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A Difficult New Test of Mechanical Comprehension * 


William A. Owens, Jr. 
Towa State College 


The Bennett Test of Mechanical Compre- 
hension is undoubtedly the prototype among 
schematic, pictorial tests which place a pre- 
mium upon mechanical “common sense” and 
insight into basic, physical principles. In a 
recent survey of V.A. Guidance Centers 
throughout the country it was reported in use 
in 90% of them, and in frequent use in 64% (1). 

Since it is so generally employed, it is only 
natural that it should occasionally be utilized 
with populations differing rather markedly 
from that for which it was originally intended. 
Under these circumstances maximally satis- 
factory results can hardly be expected. 

The writer has for some time been associated 
with a collegiate institution at which academic 
mortality in the Engineering Division is heavy, 
and in which there has been correspondingly 
great need for improved tools and techniques 
of selection. In early 1946 it became apparent 
that a revised and extended Bennett-type test, 
more schematic and more difficult than Form 
BB, might be a real asset in this context. 
Such a test has been constructed and will be 
briefly and incidentally described in what 
follows. 

Problem 


The central problem of the present investiga - 
tion is to evaluate this new Test of Mechanical 
Comprehension, Form CC, in the potentia! 
selection of engineering students. 


Materials 
Figure 1 contains typical content from the 
new test. It may be observed that there are 
five response possibilities, instead of the three 
previously employed, and that the probability 


* The writer is deeply indebted to Dr. James Wert 
for statistical assistance and advice. 


60. Which sprinkler will spin the fastest? 
(1) A, (2) B, (3) C, (4) D, (5) E. 


61. With the drive D as indicated, which picture 
correctly shows how the gears would turn? 
(1) A, (2) B, (3) C, (4) D, (5) E. 


=| be 
ey 


- FLOW 


62. Which picture shows a physical impossibility? 
(1) A, (2) B, (3) C, (4) D, (5) E. 


Fic. 1. Typical Content, Mechanical Comprehension 
Test, Form CC. 


of chance success is thereby reduced. It may 
also be noted that the drawings are quite 
schematic, unembellished line drawings which 
assume a reasonable level of sophistication. 
Like Form BB, Form CC is an untimed 
power test. 
Subjects 

The subjects upon whose scores the suc- 
ceeding results are based were all students in 
The Engineering Division at The Iowa State 
College. Freshman, Sophomore, and Senior 
classes are represented at various points, and 
the total number of cases tested is 725. 


Methods 


The data obtained were accorded the follow- 
ing types of treatments. 1. Test reliability, 
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within this relatively homogeneous group, was 
determined; and Mechanical Comprehension 
Test (M.C.T.) scores were correlated with 
scores 6n the A.C.E. Psychological Examina- 
tion to estimate the relationship of the former 
to intelligence. 2. Hierarchies of scores made 
by groups presumably differing in the ability 
in question were tabulated. 3. Correlations 
of M.C.T. scores with certain selected course 
grades were determined. 4. A random sample 
of 321 freshmen tested in 1947 has now been 
followed longitudinally for a period of three 
years to determine the relationship of M.C.T. 
score to academic mortality. 5. Biserial cor- 
relations were computed between test score 
and “A” achievement in selected courses vs. 
the balance of the achievement distribution; 
and, similarly, between test score and “F” 
failure to achieve vs. the balance of the achieve- 
ment distribution. An estimate of the effi- 
ciency of test prediction at the high and low 
ends of the achievement continuum has thus 
been obtained. 6, Multiple correlations were 
computed between the combined effects of 
A.C.E. score, high school average, and M.C.T. 
score and the criterion of selected course grades. 
Several estimates of the independent contri- 
bution of M.C.T. score have been obtained. 
7. The importance of certain background 
factors, such as farm rearing and physics 
training, in the determination of M.C.T. score 
has been estimated, through application of 
analysis of variance and covariance techniques. 
8. M.C.T. scores have been correlated with 
scores on the Kuder Preference Record in an 
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Table 1 
M.C.T. Score and Educational Status 


Inter 
Quartile 

Group Mean Range 
Agric. E. Freshmen 36 ~=—s-:15-55 
Engineering Freshmen 39 «(O55 
Archit. E. Sophomores 42 24-55 
Engineering Seniors 47 32-57 42-51 


107 
attempt to uncover relevant factors of interest 
and motivation. 


Results 


The primary results of this study have been 
summarized in Tables 1 through 7. Prior to 
more detailed comments upon these it may be 
noted that the corrected odd-even reliability 
of the new M.C.T. in a population of 321 engi- 
neering freshmen was 0.80, and that scores on 
it correlated 0.39 with scores on the A.C.E. 

Table 1, then, shows a hierarchy of M.C.T. 
scores which accords with expectation. It is 
apparent that educational attrition has elimi- 
nated many of the low scoring individuals. It 
is also suggested that the test is mainly an 
aptitude test in that, while medians rise 
through negative selection, only three of 107 
seniors scored higher than the highest entering 
freshman—and that by mere 1 and 2 point 
margins. 

Table 2 shows some representative correla- 
tions of the M.C.T. with course grades in 
various subject matter areas. Theoretical 


Table 2 


Correlations of M.C.T. Scores with Course Grades 


1. Engineering Courses 
Theoretical and Applied Mechanics 
Median grades in 7 relevant courses 
2. Shop-Drawing 
0.39 
0.32 
0.30 


Drawing and Projection 
Shop (estimated quality of product) 
Drafting 
3. Math.-Science 
Chemistry 
Algebra 


0.34 
0.28 


0.49" 
OA" 


P Engineering Group 


<.01 
<.01 


Eng. Seniors (N = 107) 
Eng. Seniors (N = 107) 


<.01 
<.01 
<1 


Arch. Eng. Sophs (N 
Ag. Eng. Sophs (N 
Eng. Freshmen (N 


177) 


260 


<1 
<O1 


Eng. Freshmen (N = 260) 


Eng. Freshmen (N = 





* These coefficients are corrected for restriction in range of test scores. 
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Table 3 


M.C.T. Scores and Mortality in Engineering 


. Passing 43 
. Failed 34 
Dropped voluntarily 3 39 
Changed divisions 57 45 40 
‘ 


Changed institutions 42 
Total = 318 


Passing vs. Failed, ry\, = 0.59; Failed vs. All Others, 
Tiw = 0.46. 


and Applied Mechanics (r=0.49) involves the 
physical science subject matters of “statics” 
and “dynamics.” Correlations with senior- 
level courses have been corrected for restriction 
in range of test scores because primary interest 
centers in the predictive value of the test as 
applied to freshman populations. Even so, 
the indicated relationships almost certainly 
provide underestimates of the corresponding 
true values, as the Registrar’s Office at Iowa 
State College allows repetition of a course with 
only the Jast grade obtained appearing on the 
official record. While it is true that the cor- 
relation of test scores with shop grades is not 
high, two circumstances seem noteworthy: (1) 
There was only one experienced instructor in 
the group of five who estimated quality of shop 
product, and the test correlated 0.48 with his 
ratings; and (2) The A.C.E. and high school 
average correlate only 0.09 and 0.11, respec- 
tively, against this same shop criterion, thus 
making it appear that the M.C.T. predicts it 
relatively well. 

Table 3 shows the relationship between the 
subsequent dispositions of 318 engineering 


Table 4 


M.C.T. Prediction by Achievement Level 


Low High 
(F's) (A’s) 

Entire vs. vs. 

Group Balance Balance N 
0.28 0.30 041 260 
0.34 0.26 0.48 260 
0.16 0.10 0.16 200 


Course 
Math. (Algebra) 
Chem. (General) 
English (Comp'n) 


Combined probability that “Low” and “High” r’s do 
not differ significantly = <.01 


freshmen tested during the Spring quarter of 
1947, and the M.C.T. scores which they made 
at that time. All but three of the original group 
are accounted for; one has been killed and two 
have records too confused to be useful. The 
Table gives the status of all subjects as of mid- 
April, 1949, at which time those listed as 
“passing” were third quarter juniors. It may 
be noted that the pass-fail biserial correlation 
which appears below the table furnishes a most 
encouraging estimate of test validity, and that 
even if all “voluntary drops” and “transfers” 
be classed as “successes” the relationship with 
test score remains substantial. 

Table 4 shows how achievement ‘level relates 
to the efficiency with which the M.C.T. will 
predict grades in a sampling of freshman 
courses. As estimated from the 3 combined 
probabilities of a real difference, it appears that 
the correlations in Column 3 are significantly 
larger than those in Column 2, and that the test 
is, therefore, a better predictor of the grades 
of the more able students than it is of those of 
the less able. Since it was originally intended, 
in the construction of Form CC, to produce a 
test which would differentiate at high ability 


Table 5 
Prediction of M.C,.T. with A.C.E. and High School Average (H.S.A.) 


(1) (2) 
A.C.E, H.S.C. 
WENN (G3 Sor Goesdec) ahs Hie eS 

(Engineering) 
Chemistry 

(General) 

Ist Qr. Av. 


Course 


0.33 0.25 


0.50 
0.55 


0.53 
0.47 


t* ” Probability of non-significant difference = <.01. 


A.C.E. 
H.S.A. 
M.C.T 


Per Cent of 
Variance 


(1), (2) & (3) 


(3) A.C.E. 
M.C.T. HS.A 

0.30 0.34" 0.39" 4-15-41 
0.34 
0.37 


0.58t 
0.58" 


0.60t 
061" 


33-47-20 
47-30-23 
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Table 6 


Characteristic 


Background Factors and M.C.T. Score 


Chance 


Highest Score F P 





Entire time 
Agriculture 


Time on farm 
Father's occupation 
Years of shop classes 
Years shop experience 
Years physics classes 
Age 


More than one 

More than one 

One half year or more 
25 years or more 


2.09 
7.00 
1.25 
0.87 
11.47** 
0.05 


>.05 
<O1 
>.05 
>.05 
<Ol 
> .05 


** Score difference, Physics vs. Non-Physics means, is 4.5 points; initial ability controlled, via analysis of 


covariance, score difference is 2.5 points 


levels, these results are as desired and suggest 
that the stated purpose has been realized. 
Table 5 shows that M.C.T. score makes a sig- 
nificant contribution to the prediction of first 
quarter grades, above and beyond those made 
in combination by score on the A.C.E. and by 
high school average. In each case the inde- 
pendent contribution of the M.C.T. is highly 
significant in spite of the fact that the criterion 
is not particularly appropriate. Column 6, at 
the extreme right, contains estimates of the 
percent of the total predictable variance in 
course grades contributed, successively, by the 
A.C.E., high school average, and the M.C.T. 
The data for Table 6 were obtained by secur- 
ing certain presumably relevant information 
about the backgrounds of 177 Agricultural 
Engineers who subsequently took the M.C.T. 
Responses to each item were assigned to one 
of two or three categories, and the test scores 
of the respondents were grouped accordingly ; 
analysis of variance was then applied to esti- 
mate the significance of the intergroup differ- 
ences in test scores. As indicated, only two of 
the six potential relationships were statistically 
significant. Confirming studies on urban- 
rural differences, subjects whose fathers were 


Table 7 
Correlations of M.C.T. with Kuder P.R 
for 260 Students 


Kuder Scales 
Me Co Se P \ L 
20" 03 .26* —.07 


Mu 


-.02 —.05 .17* 


"1% r= 0.16 


engaged in some phase of agricultural work 
did have mean test scores about 3 raw score 
points higher than those whose fathers were 
otherwise classified occupationally. Likewise, 
subjects who had had one-half year or more of 
high school physics had mean test scores ap- 
proximately 4.5 raw score points higher than 
those who had had less than this amount. It 
will, however, be immediately granted that 
there is some tendency for only the initially 
superior students to elect this difficult subject. 
Analysis of covariance was, therefore, applied 
to the results in the manner suggested by 
Lindquist (4) to control on inter-group differ- 
ences in initial aptitude as evidenced by A.C.E. 
scores. This had the effect of reducing the 
difference between the adjusted physics and 
non-physics means to approximately 2.5 raw 
score points. Such a result is interesting in 
that mechanical comprehension tests have 
sometimes been criticized for their undue de- 
pendence upon formal physics training, ap- 
parently a relatively unwarranted criticism. 

Table 7 gives the intercorrelations of the 
M.C.T. and the Kuder Preference Record. It 
will be noted that the four starred coefficients 
differ significantly from zero, but insignificantly 
from each other. Some motivational influ- 
ences may be indicated in the fact that subjects 
who are above average in scientific and me- 
chanical interest tend to score higher on the 
M.C.T. It is also interesting to observe that 
the four significant relationships identified 
would fairly well define a typical engineering 
profile on the Kuder, except for the absence of a 
computational interest. 
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Conclusions 


The following conclusions seem warranted 
relative to the role of this new test of me- 
chanical comprehension in engineering stuclent 
selection. 

1. In anengineering population, it yielded an 
odd-even reliability of 0.80 and correlated 0.39 
with the A.C.E. Psychological Examination. 

2. It yielded hierarchies of scores, by groups, 
which accorded with expectation. 

3. It showed satisfactory correlation with 
relevant course grades. 

4. It was an excellent predictor of academic 
mortality. 

5. It predicted better at high achievernent 
leve!s than at low. 

6. In prediction of course grades, it made a 
significant independent contribution beyond 
that made by A.C.E. and high school average. 

7. Score on it seemed to depend upon back- 
ground of training or experience to the extent 
of only 2 or 3 raw score points. 


8. It correlated with scores on the Kuder 
Preference Record about as would be expected 
of a test of mechanical or engineering aptitude. 


Received July &, 1049. 
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Study of Executive Leadership in Business. 
III. Goal and Achievement Index * 


C. G. Browne 


Wayne University 


The importance of the establishment of or- 
ganizational goals, purposes, and objectives; 
the need for the effective communication of 
these established goals throughout the organ- 
ization; and the evaluation of past achieve- 
ments of the organization have been recognized 
by various writers on organization and execu- 
tive activities. These factors are important 
because they are influential in decision-making, 
one of the characteristics of executive activity 
and leadership, which should be based on a 
purposive moving toward common objectives 
and which extends through all levels of super- 
visory activity. A method which can be used 
to study the effectiveness of these three func- 
tions in auy given situation among any given 
group of employees will constitute a valuable 
addition to the total study of executive leader- 
ship and the eventual! evaluation of executive 
performance. 


Procedure 


This article is devoted to the presentation of 
a method, hereinafter referred to as the Goal 
and Achievement Index, which has been ap- 
plied on an exploratory basis to the study of 
23 top executives in a tire and rubber manu- 


facturing company. Previous articles have 
presented other methods used in the study 
(2, 3). 

The subjects for the total study constituted 
all of the executives on the first, second, third, 
and fourth echelons of the business with the 
exception of one executive on the third echelon. 
They were classified into the following depart- 
mental groups: general administration, 4 cases; 
sales, 6;' finance, 4; manufacturing, 8; and 
personnel, 2. 

The detailed procedure followed in devising 
the Goal and Achievement Index form and in 

* The writer expresses his appreciation to Dr. C. L. 
Shartle for his original suggestion that a goal and 
achievement method be included in the explorations. 

‘ By his choice, the Vice-President-Sales was not 
included in the Goal and Achievement Index data. 
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accumulating the data on it has been described 
elsewhere (1). However, the general pro- 
cedure was as follows: ’ 


1. In a personal interview, each executive enumer- 
ated the goals of the company as he understood they 
existed in the planning of top management. The 96 
goals enumerated then were classified into the five 
major departments of the business to which they best 
applied. 

2. Seventeen representative statements were chosen 
and arranged in the same random order on two scales, 
one scale for the evaluation of the statements in terms 
of their importance as goals and the other for their 
evaluation in terms of the degree of company achieve 
ment in each. 

3. The executives rated the statements on a five 
point rating scale which ranged from “extremely im 
portant” to “no importance” as goals and from “com- 
pletely achieved” to “nothing done” in achievement 
terms. 

4. In calculating the individual goal and achievement 
scores, a point value of from one to five was assigned 
to each of the rating columns, the lower point value 
indicating a higher rating column, The individual’s 
score on each scale was the total points for the 17 state 
ments as he rated them, a low score indicating higher 
goal importance and degree of achievement. To deter 
mine the rank order of the statements by departmental 
groups, the individual points for each statement of the 
executives in each departmental group were added, and 
the statements were arranged in rank order on the basis 
of the total number of departmental points for the 
statements. 


Total Group Ratings 


Table 1 includes the composite ratings of the 
total group of 23 cases, with the rank order of 
each statement for both goal importance and de- 
gree of achievement indicated, and the difference 
between the rank orders. Where the difference 
between goal and achievement wasa plus value, 
it indicated that achievement in that particular 
item was considered to be above its importance 
as a goal of the company, and when the 
difference was a minus value, the opinion was 
that the item had more importance as a goal 
than the company had thus far achieved with it. 

The data in Table 1 can be analyzed in terms 
of departmental activities as indicated by the 
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Table 1 


Rank Order of Composite Goal and Achievement Ratings for Total Group of 23 Executives 


2 3 
Goal Ach 
Rank Rank 
Statement of Purpose Order Order 
. To create big production ) 14 
. To make as much money as possible for the stockholders. P 13 
. To continue to keep Congo a successful and prosperous 
company. 
. To establish a more definite and concrete sales policy. 
. To develop a better community and help the general 
prosperity of the city. 
. To broaden its field of activity by making a wider variety 
of products. 
. To make as fine a tire as any company can make and 
sell it a little cheaper. 
. To promote good labor relations and have satisfed 
workers. 
. To make a fair margin of profit. 
. To make a good and economical tire and sell it to private 
brand, mass distributors at a good price. 
. To increase the proportionate volume of Congo brand 
business handled 
. To develop an extensive, national advertising program. 
. To keep the company growing and expanding 
. To expand the sales force in the field. 
. To come out on top with a good quality tire and high 
production. M 
. To provide good working conditions and good living 
standards for employees. P 
. To attempt to cut costs and expenses and put the com- 
pany back in the profits brackets F 2 


* Department Code: GA—-General Administration; F— Finance; S-—-Sales; M— Manufacturing; P-— Personnel 
The department designated for each statement refers to the department of the company to which the statement 
best applied in the operational activities of the company. 


departmental classification of each item, and 
also individual goals can be studied without 


A study of the rank order of the statements 
by goal importance and degree of achievement, 


regard for their departmental classification. 
Ideally it could be hoped that the rank order 
of the statements in terms of goa! importance 
would be the same as the rank order of the 
statements in degree of achievement terms. 
This agreement would indicate a balance be- 
tween these two important areas of executive 
activity and leadership. However, where dif- 
ferences are shown to exist, the opportunity is 
offered to top management to study the ad- 
visiability of establishing policies which will 
restore a balance by concentrating executive 
attention and effort on those activities which it 
has been indicated were not in balance. 


without consideration of the differences be- 
tween the ranks, will provide information to 
top management which may be used in at- 
tempting to re-educate management in general 
or which will provide the basis for conference 
and consultation between management at 
various echelons regarding the proper place 
of various goals in the business and the extent 
to which these goals have been achieved. 

For purposes of illustration here, those state- 
ments relating to the finance department (nos. 
2, 9, and 17) and those relating to the personne] 
department (nos. 5, 8, and 16) will be examined. 
Statement 9 regarding a fair margin of profit 
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had a minus 11 difference, meaning that these 
executives believed the company had fallen 
far short of the great importance of this goal. 
In contrast to this, however, statement 2 ‘to 
make as much money as possible for the stock- 
holders” received a plus two difference, indi- 
cating that these executives believed the com- 
pany achievements had somewhat exceeded the 
goal importance of this activity. The remain- 
ing finance item with a minus one difference 
between ranks indicated that the company was 
working quite satisfactorily towards cutting 
expenses and putting the company back in the 
profits bracket, the successful completion of 
which should change the rank differences on 
statement 9. 

When the statements relating to personnel 
activities were examined, it was noted that all 
three of the statements received a plus differ- 
ence, meaning that in all of these activities the 
executives believed the company had achieved 
more than the goal importance of these per- 
sonnel functions. The sensitivity of the Goal 


and Achievement Index form may be judged 
somewhat by the fact that, at the time the 
study was conducted, tire orders were falling 
off rapidly; the company was not operating at 


a profit; and it was the general executive 
opinion that the company could not re-establish 
itself on a profit basis in the highly competitive 
tire market until labor costs were reduced. It 
was also known that the union contract was 
about to come up for renegotiation and that 
in all probability there would be a demand for 
higher wages which would eliminate the possi- 
bility of decreasing labor costs. 


Communication and Departmental Group 
Correlations 


Table 2 gives the rank order correlations 


between goal and achievement ratings of the 
five departmental groups; and a composite of 
the president and general manager, vice presi- 
dent manufacturing, and treasurer (herein- 
after referred to as PVPMT). The correla- 
tion of .47 for the total group actually was the 
rank order correlation between columns 2 and 
3in Table 1. Similar tables were prepared for 
the rankings of each departmental group, and 
the rank order correlations computed from 
them. The correlations indicated the extent 


Table 2 


Rank Order Correlations Between Goal and Achieve- 
ment Composite Ratings of the 17 Items 
by Departmental Groups 


PVPMT Mfg. Sales 
.22 49 A7 


Genera! 
Finance Adm. Personnel 


28 66 64 


of agreement among executives in a given de- 
partment on the relationship between the im- 
portance of company goals and the degree of 
achievement.? 

Communication refers to how well the 
policies and thinking of top management have 
been carried through and absorbed by the re- 
mainder of the organization. If there were a 
condition of perfect communication within the 
company on the questions of goal importance 
and degree of achievement on the type of state- 
ment included in the Goal and Achievement 
Index form, then it could be expected that the 
ratings of each departmental group on all of the 
items would be identical with the ratings of the 
PVPMT, since they represent top manage- 
ment. If the ratings by departmental groups 
were the same, the rank order of each state- 
ment would be the same for each department, 
and consequently the rank order correlations 
of the various departments would be equal. 
These identical correlations might vary from 
minus 1.00 to plus 1.00, but as a measure 
of communication only, the size or the sign 
of the correlations bears no relationship to 
the effectiveness of communication. Instead, 
the effectiveness of communication can be 
measured only by observing the differences in 
the size of the correlations between various 
departmental groups and top management. 

Therefore, the correlation of .66 for the 
general administration group expressed the 
highest group ratings toward the degree of 
achievement in terms of the goal importance 
of the various items. However, this correla- 
tion expressed the greatest disagreement with 
the opinion of the PVPMT and, therefore, 
indicated the least adequate communication. 

2 It should be noted that these correlations were de- 
scriptive of the relationships existing in this population 


of executives only, and cannot be considered as sam- 
pling statistics. 
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On the other hand, the correlation of .17 for 
the sales department, although it expressed the 
lowest departmental ratings toward the degree 
of achievement in terms of the goal importance 
of the items, was in closest agreement with the 
ratings of the PVPMT. 


Analysis of Individual Statements 


Although correlations and composite ratings 
are valuable for the general or over-all picture 
which they indicate, correlations obscure much 
detailed data which are available in the Goal 
and Achievement Index form and which can 
be analyzed by comparing individual state- 
ments in terms of departmental ratings, 
echelon levels, and individual executives. For 
the purpose of this article, one of the state- 
ments will be considered in terms of the differ- 
ences in its ratings inter-departmentally. 

On statement 10 (Table 1), there was a 
range of 11.0, the rank extending from 3.5 to 
14.5. This statement was specifically im- 
portant to a long-time comPany policy of selling 
tires under a private brand name (not Congo) 
to large retail outlet chain’ stores. The 
PVPMT gave this goal a 3.5 rank, indicating 


that this policy should be continued, and per- 
haps even emphasized. However, the sales 


department ranked this statement 14.5. This 
may have meant one of two things: (1) that 
the sales group was not aware of top manage- 
ment attitude toward this policy or (2) that the 
sales group was expressing its own opinion on 
the policy without regard to what its know- 
ledge of top management attitude was. Under 
any possibility, statement 10 was classified as a 
sales statement regarding which the sales group 
deviated widely from top management attitude. 
The implications of this condition for the 
efficient operation of the company and the 
performance of the executive leadership within 
the company may be many. 

When the statements were considered inter- 
departmentally in terms of achievement ranks, 
the highest range of 13.0 was for statement 1, 
classified as a manufacturing statement. The 
rankings extended from 3.0 to 16.0. The 
manufacturing group and the PVPMT gave 
this a 3.0 achievement rank, indicating their 
belief that the degree of achievement was well 
near the top of company achievements. How- 
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ever, the personnel group which reported to 
the vice president manufacturing ranked the 
statement 16.0, and the sales group ranked it 
next lowest with a 13.0. 

Throughout these detailed comparisons, in- 
formation was obtained which provided clues to 
the relationships existing inter-departmentally 
between executives in the business. One pat- 
tern which developed was a general tendency 
for executives in any given department to 
rank a statement which applied to their de- 
partment in the higher ranks obtained by that 
particular statement. For example, a state- 
ment which applied to the sales department 
was likely to be ranked higher both for goal 
importance and for degree of achievement by 
the sales executives than by any other depart- 
mental group. 


Rankings by Departmental Statements 


In terms of efficient company and executive 
relationships as well as providing a measure of 
communication and knowledge of company 
goals and achievements, the differences be- 
tween ranks assigned to statements grouped 
departmentally provided additional leading in- 
formation. For example, when it was found 
that items which were given little or no goal 
importance were given high degree of achieve- 
ment, or vice versa, then it appeared evident 
that there was a need for corrective activity 
in the particular aspect of company affairs 
covered by the item. 

Table 3 presents the average rank differences 
between goal importance and degree of achieve- 
ment rankings of the statements grouped 
according to the department to which they 
best applied. The rankings were the compo- 
site of each departmental group of executives. 
A plus value indicated that the degree of 
achievement for any given group of depart- 
mental statements as ranked by any given 
group of departmental executives was greater 
than the goal importance of the same group of 
departmental statements as ranked by the 
same group of departmental executives. A 
minus value, on the other hand, indicated that 
the degree of achievement for any given group 
of departmental statements as ranked by any 
given group of departmental executives had 
not reached the goal importance of the same 
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Table 3 


Average Rank Differences Between Goal Importance and Degree of Achievement Rankings of Departmental 
Groups of Executives on Goal and Achievement Statements Classified into 
the Department to Which They Best Apply 


Finance 
Statements 
+ 


Genl. Adm. 
Statements 


3.3 
2.3 
1.0 
37 
3.0 
43 
4.2 


PVPMT 
Mig. 

Sales 
Personne! 
Finance 
Gent. Adm. 


group of departmental statements as ranked 
by the same group of departmental executives. 

It will be noted that, with the exception of 
the personnel statements, there were only four 
cases of plus values. That is, the executives, 
regardless of department, believed that the 
company had over-achieved in those items 
which relate to personnel activity but in other 
activities there was almost consistent agree- 
ment on under-achievement. The personnel 
executives with an average plus difference in 
rankings of 4.0 expressed the smallest over- 
achievement value on personnel statements. 
On the horizontal rows, the manufacturing 
executives gave their lowest under-achievement 
value to manufacturing statements, and the 
sales executives gave a plus value or an over- 
achievement ranking on sales statements. On 
the other hand, however, the finance executives 
ranked their own departmental statements 
fifth or last. This attitude on their part may 
have been indicative of a somewhat cautious 
or pessimistic attitude on the part of those men 
who were guardians of the company purse, or it 
also may have been a direct reflection of the 
particularly acute financial situation in the 
company at that particular time. 

Again in this phase of the study, there was 
evidence which suggested that the rankings 
given to goal and achievement statements were 
related to (1) the department to which the 
executive was assigned and (2) the type of 
work which the executive did. 


Individual Goal and Achievement Scores 


The individual goal and achievement scores 
and the three scores for each executive based 


Statements 
4. 


Manufacturing 
Statements 


Personnel 
Statements 
+ 

65 Al 
8.2 2.4 
6.5 4 
62 29 
4.0 0.0 
90 = 
42 


on the relationship between the goal and 
achievement scores are included in Table 4.5 

For the G/A calculations a score of 1.00 in- 
dicated no difference between goal and achieve- 
ment scores. In other words, the individual 
with a 1.00 G/A score ranked the 17 state- 
ments for degree of achievement of the com- 
pany equal to the goal importance of the state- 
ments. Therefore, it follows that a G/A score 
of less than 1.00 indicated that the degree of 
achievement of the company had not equalled 
the goal importance of the statements, and a 
score of more than 1.00, that achievement had 
surpassed goal importance. 

The range of the G/A scores was from .480 
to 1.00. With the exception of the one case 
who ranked goal importance and achievement 
equal, all of the executives ranked achievement 
below goal importance. 

The G minus A scores represented the al- 
gebraic difference of the goal score and the 
achievement score for the individual. All of 
the G minus A scores were minus, ranging 
from 0 to —28. This indicated that the degree 
of achievement had not reached goal impor- 
tance, with the exception of the one score of 0 
which indicated equality between achievement 
and goal importance. In this study it was 
somewhat simpler to work with the G minus 
A score since all of these scores were minuses 
or zero. If there were plus scores included, it 

* To reduce printing costs, Table 4 has been deposited 
with the American Documentation Institute and may 
be ordered as Document 2741 from American Docu- 
mentation Institute, 1719 N Street, N.W., Washington 
6, D. C., remitting $0.50 for microfilm (images 1 inch 
high on standard 35 mm. motion picture film) or $0.50 


for photocopies (6 X 8 inches) readable without optical 
aid. 
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would then simplify the study to work with 
the G/A score. 

Both the GA and G minus A scores can be 
used to measure the ratings of individuals to- 
ward company goals and their relationships 
to achievements of the company. However, 
to measure communication and knowledge of 
company goals and achievements as expressed 
by top management, some other score must be 
used. For this purpose, the algebraic differ- 
ence between the individual executive’s G 
minus A score and the G minus A score of the 
president and general manager was taken. All 
of these scores were plus values with the ex- 
ception of three cases. This indicated that, 
with the three exceptions, all of the executives 
ranked the company’s achievements in relation 
to goal importance greater than the president 
and general manager. did. As a measure of 
communication, however, it was immaterial 
whether the individual’s disagreement with 
the president and general manager was a plus 
or a minus score. The sign merely indicated 


the trend of the difference in terms of goal 
importance and achievement relationship, but 
a minus score had the same communication 
meaning as a plus score. 


As another phase of the study, the G minus 
A scores and the G minus A deviations from 
the president and general manager were con- 
sidered, arranged first by departmental groups 
of executives and then by echelon groups. The 
analyses of these results were characterized by 
two factors: (1) the executive contacts which 
the individual had within the business, and (2) 
the extent to which the work performed by 
the executive was related to the major over-all 
functions of the business. While this general 
pattern could not be supported and justified 
with each individual executive, the tendency of 
the pattern was strong enough to justify the 
recommendation that it be pursued further in 
other research. 


Summary 


The results of an exploratory study with a 
method for studying executive leadership 
called the Goal and Achievement Index have 
been presented, the data being based on a 
group of 23 executives in a tire and rubber 
manufacturing company. ‘The data were ana- 
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lyzed for the total group of executives, for the 
executives grouped into the major departments 
of the business, and for individual executives. 

On the basis of this exploratory study, it 
appears that the approach used in the Goal and 
Achievement Index offers promising possibi- 
lities for the study of executive function and 
leadership in business as it is related to (1) 
knowledge of company goals and achieve- 
ments; (2) the relationship between goals and 
achievements; and (3) the effectiveness of the 
communication of this knowledge and relation- 
ship from top management through the super- 
visory levels of the business. 

In the analysis of goal and achievement 
scores, two fundamental variables need to be 
particularly considered. The first has to do 
with the goal and achievement statements and 
the individual’s evaluation of them at any given 
time. This variable is a constantly changing 
one, subject to variation in accordance with 
the conditions existing within a particular 
business at a particular moment. 

The second variable is the relationship ex- 
isting between the scores and ratings, inter- 
individually and inter-departmentally. It can 
be postulated that this relationship will remain 
relatively constant, unless there are changes 
in the personnel studied or unless the company 
attempts some type of a training program 
which will bring the executives, individually 
and departmentally, into greater agreement 
on and provide better understanding of com- 
pany goals and achievements. Therefore, this 
factor is not subject to the same variability as 
the evaluation of individual goals and achieve- 
ments since it is dependent upon the executives 
as individuals and not upon the constantly 
changing factors in company operations. 


Received August 8, 1949. 
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To What Extent Have the American People Accepted Socialism? 


Henry C. Link and Albert D. Freiberg 
The Psychological Corporation, New York City 


In 1928, the national convention of the 
Socialist Party adopted a platform including, 
among others, the following domestic planks: 

“1. Public ownership of all national re- 
sources and public utilities, including oil wells, 
coal mines, power plants, railroads, and tele- 
phone and telegraph facilities. 

2, Governmental relief of unemployment by 
extension of all public works, Federal loans 
without interest to states and municipalities to 
promote public works, old age pensions, un- 
employment, health, and accident insurance, a 
shorter work day. 

- 3. Acquisition of grain elevators, stock 

yards, and other distributing agencies by the 
government or by bonafide cooperating socie- 
ties; cooperative purchasing, marketing, and 
credit agencies; government insurance against 
crop losses; flood control.’” 

The intervening years have seen the enact- 
ment of some of the Socialist goals and the 
present administration has recommended legis- 
lation which would, in part, achieve others. 
The attitudes of present day citizens toward 
Socialism, their definition of socialistic aims, 
and their opinions of existing or proposed 
_ activities which might be regarded as imple- 
mentation of socialistic philosophy constitute 
an interesting area for opinion research. 

In a previous article,? we have reported the 
finding that 67% of an urban sample regarded 
Communism as a “‘dangerous thing in the 
United States” while 26% of the same group 
regarded Socialism as dangerous. Sixty-one 
percent of the respondents felt that Commu- 
nism and Socialism were different. 

In October, 1949, a random subsample of 
1000 adults out of the 5000 interviewed in con- 
nection with the Psychological Barometer were 
asked a series of questions designed to yield a 
more detailed account of opinion on this topic. 

' Socialist Party (U.S.) Socialist Campaign, 1928 
As reported in the WORLD ALMANAC of 1929, the 
Resale National Convention of 1928, specifically 
endorsed none of these demands except for a mention 
of cooperative farm marketing agencies. The Demo- 
cratic Convention favored government support (but not 
subsidy) of cooperative farm marketing agencies, public 
works to relieve unemployment, and flood control 

* Link, H. C., and Freiberg, A.D. The Psychological 
Barometer on Communism, Americanism and Socialism. 
J. appl. Psychol., 1949, 33, 6-14. 
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Q. Are you for or against Socialism in this country? 


Answers % 
Against Socialism 
For Socialism 
Don’t Know 


75.3 

6.3 

18.4 

By the various socio-economic 
answers were as follows: 


groups, these 


Upper Lower 
Answers Upper Middle Middle Lower 


% 
64 
8 


28 


oF 
4 


Against Socialism 90 
For Socialism 3 
Don’t Know 7 


oO Oo 


To 70 
80 74 
6 7 


14 19 


This was followed by: 


Q. What does Socialism mean to you? 
The 75% who were against Socialism and the 6% for 
Socialism described it as follows: 


% 
Answers Against 


Government ownership or contro! of in- 


dustry, utilities, natural 
health and 
doctors 


Too much Government control; dicta- 


resources, 


welfare, medicine and 


torship; regimentation 

Distribution of wealth; fellow who pro- 
duces gets no more than the one who 
doesn’t 

Destruction of capitalism; restricting or 
doing away with private enterprise 
and competition; disruption of our 
economic system; ruin of our country 

Socialism is related to Communism, leads 
to Communism 

Equality for all; no racial discrimina- 
tion; a hopeless crusade for equality 

Socialism is like the English government 
of today 

Workmen’s compensation; social secu- 
rity; the “Welfare State” 

Collectivism; collective 
planned economy 

Miscellaneous answers 

Don’t know 


Government; 


Total Answers 


Total Interviews (Base for per cents 
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Evidently the majority who say they are 
against Socialism tend to identify Socialism 
with increasing Government control, operation 
and ownership. 


Specific Issues 


In order to bring the general issue of Social- 
ism down to specific cases, the following series 
of questions was then asked of everyone in our 
1000 interview sample. 


Which of the following do you think are steps 
toward Socialism, and which are not? 

The minimum wage law where the Government 
sets a minimum wage below which no (interstate) 
business can pay? 

The T.V.A. and other Government-owned flood 
control and electric power projects? 

Payroll taxes or deductions from a person’s pay to 
provide old age, unemployment and other benefits? 
Food subsidies where the Government buys eggs, 
potatoes, and other farm products to keep their 
prices near parity? 

Rent control where the Government decides how 
much rent can be asked for houses and apartments? 
Government housing where the Government builds 
apartment houses which are rented to low income 
people at rents below actual cost? 

Government rules which require that only union 
labor can be used on Government building? 
Peace-time price control where the Government 
decides how much a business can ask for its 
products? 


The responses of those 753 persons who pre- 
viously had reported themselves as opposed to 
socialism are summarized at topof next column. 


Conclusions 


Although 75% of a sample of urban adults 
represent themselves as opposed to Socialism, 
within this 75% each of eight types of govern- 
ment activity which might logically be regarded 
as a specific instance of socialistic practice finds 
a considerable number who favor it and a 





Yes, 
It Is 
Socialistic 


Items 


a. Payroll taxes for old age and 
other benefits, etc. 
. T.V.A. and similar projects, etc. 
:. The minimum wage law, etc. 
. Government housing, etc. 
Rent control, etc. 
Covernment rules requiring 
union labor on Government 
building, ete. : 19 
. Food subsidies for farm prod- 
ucts, etc. 5: 24 
h. Peace-time price control, etc. 7 18 





number who do not regard the practice as “a 
step toward Socialism.” 

Four of these areas, payroll taxes for old 
age benefits, T.V.A. and flood control, mini- 
mum wage laws, and government housing find 
favor with a majority of those who oppose 
Socialism in general. Furthermore, in the case 
of each of these categories fewer than half of 
those against Socialism regard the measure as 
socialistic. 

The more frequently a specific type of ac- 
tivity is regarded as Socialistic, the less fre- 
quently is it viewed with favor. 

Some of the planks of the 1928 Socialist 
platform have been translated into practice. 
Several of these are not now regarded as “‘steps 
toward Socialism” and are favored by a ma- 
jority of those adults who say they are against 
Socialism in the United States. 

Even though Socialism may be a scare-word 
to 75% of the people, it does not follow that 
this fear will be translated into the fear of 
specific measures such as Government owner- 
ship of public utilities, government housing, 
“socialized” medicine and the like. 


Received January 18, 1950. 
Early publication. 














The Selection of Patrolmen 


Philip H. DuBois and Robert I. Watson 
Washington U niver sity 


Relatively little is known concerning selec- 
tion of police officers by psychological methods. 
In a recent exhaustive review of the validity 
of commonly employed occupational tests, 
Ghiselli' reports briefly on previous work. 
Without either specifically identifying the 
particular tests utilized or the studies from 
which they came, he presents certain correla- 
tions with criteria of job proficiency. The 
median coefficients of validity with various 
kinds of tests were as follows: intelligence .28, 
immediate memory .40, arithmetic —.12, 
number comparison .25, name comparison .35, 
finger dexterity .20, and personality question- 
naires .24. With the exception of intelligence 
in which eight coefficients were involved, most 
of these validities were based upon one or two 
correlations. The total combined N in all the 


studies numbered less than 500. The magni- 
tude of the correlations, even in the absence 
of information about their reliability, shows 


that this field of personnel selection has 
certainly not yet been satisfactorily explored. 
With Ghiselli it is agreed that ‘‘about all that 
can be said with any degree of certainty is that 
intelligence tests give fair results and that 
personality measures would appear to be 
promising.’ 

The present study on the prediction of suc- 
cess of patrolmen in training and on the job 
was undertaken at the request of the St. Louis 
Board of Police Commissioners. In late 1947 
an extensive reorganization of the Police De- 
partment was in progress. New equipment 
was secured, new office procedures were insti- 
tuted, the crime detection laboratory was re- 
organized and expanded, a Police Academy for 
recruit and in-service training was organized, 
and plans were made for the introduction of 
psychological methods for the selection and 
evaluation of personnel. As a first step in the 
application of psychological methods to per- 

'Ghiselli, E. E. The validity of commonly em 
ployed occupational tests. Univ. Calif. Publ. Psychol, 


1949, 5, No. 9, 253-288 
* Op. cit., p. 269 


sonnel problems, the authors were invited to 
participate in the selection of classes of , 
bationary patrolmen. 


The Sample 


Table 1 presents information on the size and 
composition of the sample at various steps in 
the procedures. It will be noted that 1255 
applications for appointment were processed. 
It was found, however, that only 512 of the 
men met the various requirements set up by 
the Board, as described in Table 1, as to 
education, age, height, weight, residence, 
citizenship, and satisfactory character. As 
many of the applications had been on file for 
some months, these 512 men were canvassed 
as to their continued interest in being appointed 
to the Force. The 312 still interested were 
invited to take the selection tests, with 253 
actually appearing for the examination. 

Of these, 180 were found to meet the selec- 
tion criteria set up for the psychological test 
battery shortly to be described. Twelve men 
were eliminated by failure to pass the physica! 


Table 1 
Derivation of the were Studied 


Applications for appointment 

Applicants meeting established standards 
(Height 59” to 6'4”; weight 150 to 210 Ibs.; 
age 22 to 31 years with year-for-year waiver 
for military service; residence 2 years in city; 
citizenship; education 10th grade or equiva- 
lent; good character as shown by police inves- 
tigation) 

Applicants requesting examination 

Applicants appearing for examination 

Examinees meeting cutting scores 
(AGCT score of 100 or more, CWF-2 of 9 or 
less) 

Passed physical examination 

Passed by Screening Board 

Entered Classes 1 and 2 

Sample used in-present study 
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examination, leaving 168 who were interviewed 
by a Departmental Screening Board. The 
Screening Board consisting of five members of 
the Department, including the Chief of Police, 
rated each man on five traits: appearance, 
manner, speech, adaptability, and general im- 
pression, giving scores of 1 for the lowest and 
5 as the highest score in each of the character- 
istics. A score of 125 was perfect and a score 
of 75 or above was considered acceptable. 
Twenty-nine candidates were eliminated on the 
basis of the screening board rating. The re- 
maining 139 were appointed as probationary 
patrolmen, entering the Police Academy in two 
successive eight-week classes, which graduated 
in June and August, 1948. The present study 
is concerned with 129 of these 139 cases. Of 
the ten cases not being reported, two resigned 
while in good standing at the Academy, five 
failed in the curriculum, while in the three 
other instances data were incomplete for one 
reason or another. The number of failures in 


training was not large enough for consideration 
in this paper, although the problem is scheduled 
for study at a later date. 


Tests Administered 


The battery of psychological selection and 
experimental tests used in this study is listed 
in Table 2. Before turning to the results 
each instrument will be described. 

The St. Louis Police Aptitude Test was 
developed by the writers in consulation with 
the departmental staff. All material was 
prepared in the police context, so that it had 
high face validity. Realism was obtained by 
using police photographs and excerpts from 
police manuals. The items were screened by 
experienced police officers who decided whether 
or not they were plausible in a police setting. 
In its final form the test consisted of 90 
multiple choice items divided into five sections. 
The first section involved 15 memory items. 
They were of two sorts: memory for photo- 
graphs and names, and memory of a scene of a 
crime and of a scene of an accident. A touch 
of verisimilitude was added by using the pho- 
tographs of recent criminal suspects. The 
names were selected at random. The accident 
and crime scenes were posed and involved in 
each instance a variety of clues. Typical of 


the questions asked was one involving the 
locus of the crime. The second section of 15 
items required the applicant to select the one 
misspelled word from a list of five or to select 
the one correctly spelled from a list of words 
otherwise incorrectly spelled. Typical words 
included alias, waiver, accomplice and felony. 
Three reading selections made up the third 
section. These were reports of a fire, a rob- 
bery, and a selection from the police manual 
concerning rules in regard to the making of 
arrests. Each was followed by five multiple 
choice reading comprehension items. The 
next section consisted of 30 items concerned 
with genera! information and judgment related 
to police work. Police practice, causes of 
crime, the purpose of teaching first aid to 
policemen and the reason automobiles need to 
be inspected regularly were typical of the bases 
for these questions. Care was taken that 
specialized knowledge generally known only 
to a person with police experience was not 
required. The last section involved 15 arith- 
metical problems couched in a police depart- 
ment setting. The test as a whole required 90 
minutes, including directions. Originally it 
was proposed to use a cutting score with this 
test, but since the correlation with the AGCT 
was higher than anticipated it was used only 
experimentally with the first two classes. 

The Army General Classification Test, First 
Civilian Edition, was actually used for selec- 
tion. An AGCT score of 100, corresponding to 
a percentile rank of 48, was the minimum 
acceptable. The two classes had a mean 
AGCT score of 118 or a percentile rank of 80 
in the general population. This score stands 
at the 75th percentile of the distribution re- 
ported for policemen. 

Candidates were asked to write a short com- 
position entitled ‘“‘Why I Wish to Become a 
Policeman.” This essay was graded for hand- 
writing, using the Thorndike Handwriting 
Scale, and for coherence in English expression, 
using a five-point scale. No candidates were 
actually eliminated on this test but the lowest 
rating essays of men who were otherwise 
qualified were scanned by the Chief of Police 
for acceptability. The final test included in 
the original screening battery was the Cornell 
Word Form—-2, a controlled association test 
which has been used to detect neurotic ten- 
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dencies. Scores of ten or more were con- 
sidered disqualifying. 

In the course of the eight weeks training at 
the Police Academy it was possible to ad- 
minister for experimental purposes a further 
series of tests. These included the Figure 


Matching Test, a speeded perceptual measure 
involving matching the silhouette of an object 
or a meaningless shape with five similar figures, 
one of which is the exact duplicate; the Bennett 
Mechanical Comprehension Test--Form BB; 
the Minnesota Paper Form Board MB; the 
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Object-Aperiure Test, an instrument requiring 
visualization in three dimensions and yielding a 
speed score and a power score; the Strong 
Vocational Interest Blank, which was com- 
pletely scored but which has been validated for 
only three scales—police interest, interest 
maturity, and occupational level; and, finally, 
the Rosenzweig Picture Frustration Study, 
developed as a measure of the direction of 
aggression and the type of frustration reaction. 

The Strong and the Rosenzweig measures 
were introduced to study the validities of 


Table 2 


Validities of Psychologica! Tests and Screening Board Rating for Prediction of Success as Probationary 


Patrolman (N = 129). 





Academy 
Grade 
June 
Class 
(N = 72) 
Police Aptitude Test-—Total 
Aussage (15 items) 
Spelling (15 items) 
Reading (15 items 


ao" 
44" 
18 
DS ees 
31 ** 
54** 
53°* 
41 ** 
37** 
03 
23* 
14 
25* 
28* 
ma 
18 
28" 
09 
14 
06 


Information and Judgment (30 items 
Arithmetic (15 items) 
Army Gen. Classif. Test--Total 
Verbal Items 
Numerical Items 
Blocks 
Handwriting 
Composition 
Cornell Word Form 2 
Figure Matching (speeded perceptua! 
Bennett Mech. Comp. BB 
Minn. Paper Form Board MB 
Object-Aperture Test 
Object-Aperture Test 
Strong Voc. Int 
Strong Voc. Int. 
Strong Voc. Int 


Speed 
Power 
Police Interest 
Interest Maturity 
Occupational Level 
Rosenzweig Picture Frustration 
Obstacle Dominance 
Ego Defensiveness 
Need Persistance 


— 03 
06 
O4 

Impunitiveness 

Intropunitiveness 

Extrapunitiveness 

% Group Conformity Rating 
Screening Board Rating 


** Significant at 1% level 
* Significant at 5% level 


39°" 


Classes of June, 1948, and August, 1948, of Police 
Academy, Department of Police, City of St. Louis 


Academy , 
Grade 
August 
Class 

(N = 57) 

5o** 
20 
42** 
40°" 
37°* 
bi 
sr 
§3°* 
43** 
a 
04 
30° 
-.18 


Achieve 
ment 
Test 

(N = 129) 
ho 

as” 
38** 
41 »“* 
3o"* 
42"* 
47** 
aac 
36** 
23** 
06 
26** 
22° 
23** 
20° 
3o"* 
09 
17* 
24°** 
05 
sor 


Service 
Rating 
(N = 129) 
03 
.O8 
~ 03 
~ 03 
12 
~ O07 
10 
03 
02 
14 
10 
08 
.06 
03 
10 
04 
10 
12 
~ O1 
04 


Marksman- 
ship 
(N = 129) 
08 
20* 
10 
11 
01 
i4 
15 
02 
Al 
16 
.06 


29" 
20" 
14 
20" 
12 
17 


~11 
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certain interest and personality characteristics 
for police work. The other experimental tests 
were non-verbal instruments which were con- 
sidered to be possibilities for supplementing the 
selection tests, success in which depended 
largely upon verbal ability. 
Validities 

Four criteria of validity are used in this 
study: the final grade in the Police Academy, 
treated separately for the two classes; an 
achievement test based on Perkins’ “Elements 
of Police Science,” which was required reading 
after completion of the course and on which 
the patrolmen took an examination approxi- 
mately 10 weeks after graduation; marksman- 
ship during the academic training; and service 
rating after 10 weeks on duty. The service 
rating was based upon an 11-trait rating scale 
with five descriptive steps for each trait. The 
traits were work attitude; loyalty, interest and 
enthusiasm; judgment; report writing; inves- 
tigative ability; alertness; bearing and de- 
meanor; speech; appearance; contacts with the 
public; and usefulness to the service. Ratings 


were made by the sergeant who directly super- 
vised the work of the patrolman and were ap- 


proved by superior officers in the chain of com- 
mand. Only the validities against the over-all 
service rating are reported at this time. 

In Table 2 validities are reported for all 
predictor variables against these four criteria. 
It will be noted that for the criteria of academy 
grades and the achievement test, the verbal and 
numerical sections of the Police Aptitude Test 
and Army General Classification Test have the 
highest validities, closely followed by the Com- 
position Test and the non-verbal experimental 
tests. For 72 cases reported in the June Class 
there are ten correlations significant at the 
1% level and five more at the 5% level. For 
the August class of 57 cases, there are seven 
correlations significant at the 1% level and 
seven more significant at the 5% level. 
Against the Achievement Test, where the N is 
129, there are 15 correlations significant at the 
1% level and four more significant at the 5% 
level. Against the criterion of marksmanship 
there are only two correlations significant at 
the 1% level and four at the 5% level. It will 
be noted that except for one of the Rosenzweig 


categories, all the correlations which are signifi- 
cant against the marksmanship criterion are 
non-verbal aptitude measures. None of the 
individual correlations with service rating is 
statistically significant at the 5% level. 

The negative validities of the Police Interest 
Scale of the Strong Vocational Interest Blank 
against all criteria except marksmanship are 
perhaps surprising. However, the academy 
grade and the Achievement Test would prob- 
ably be somewhat artificial criteria of perfor- 
mance for policemen on whom the scale was 
developed. A tabulation of the Police Interest 
Scores of the men in the first class showed 
86° of them A or B+, with no score below 
B—. It is possible that in a greater range of 
interest scores, the test would have been 
predictive. 

The complete matrix of approximately 1000 
intercorrelations of all the predictor variables 
and the several criteria of success has been 
computed. The intercorrelations of 22 selected 
predictor variables are presented in Table 3.* 
The Police Aptitude Test has a correlation of 
.53 with the Army General Classification Test 
in this restricted range. The intercorrelations 
of the non-verbal experimental tests ranged 
from .20 to .55, indicating that there is con- 
siderable over-lap among them. 

In Table 4, Beta weights, together with 
multiple correlations, are presented for the pre- 
diction of several criteria of success. For 
academic grades and achievement tests, the 
Wherry-Doolittle test selection method was 
employed. For marksmanship and _ service 
ratings ordinary regression weights were com- 
puted using the variables which appeared to 
be the most promising. 

It is immediately apparent that the com- 
binations of tests useful in prediction differ 
considerably from criterion to criterion. The 
Police Aptitude Test, the Army General Clas- 
sification Test and at least one of the non- 
verbal tests (Minnesota Paper Form Board or 
Bennett Mechanical Comprehension) appear 
in the regression equations predicting the 

* To save space, Table 3 has been deposited with the 
American Documentation Institute and may be ordered 
as Document 2696 from American Documentation In- 
stitute, 1719 N Street, N.W., Washington 6, D. C., 
remitting $.50 for microfilm (images 1 inch high on 


standard 35 mm. motion picture film) or $.50 for photo- 
copies (6 X 8 inches) readable without optical aid. 
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Table 4 


Beta Weights and Multiple R’s in Predicting Several Criteria of Success as Probationary 
Patrolman, St. Louis Police Department 





Academic 
Grade 
Class 1 

(N = 72) 


Academic 
Grade 
Class 2 

(N = 57) 


Achieve- 
ment 
Test 

(N= 129) 


Marksman- 
ship 


(N = 129) 


Service 
Rating 
(N = 129) 





Police Aptitude Test Total - All 
Aussage 
Information and Judgment 
Army Gen. Classif. Test--Total 
AGCT—Blocks 
Composition 
Figure Matching 
Bennett Mech. Comp.—BB 
Minn. Paper Form Board MB 
Object-Aperture (Power) 
Strong—Occupation Level 
Rosenzweig 
Ego Defensiveness 
Intropunitiveness 


i 
| Multiple R 


28 .28 
12 


25 





** Significant at 1% level. 
* Significant at 5% level. 


_“academic”’ criteria, i.e., academic grade in 
Class 1 and Class 2, and the achievement test. 
- The composition test carries a weight in Class 
2 and the Strong-Occupational Level carries a 
weight for the achievement test. 

Marksmanship can be predicted to some de- 
gree from tests of a non-verbal sort, involving 
perception, space relations, and mechanical 
comprehension. 

Service rating, which is predicted least 
accurately, depends to a considerable degree 
| upon personality predictors, with some weight 
being carried by the tests of spatial relations 
and information and judgment. For service 
rating the multiple of .29 is significant at the 
5% level of confidence, while all other multiple 
R’s are significant at the 1% level. It may be 
that tests that will predict the service rating 
more adequately will have to be devised. 

A third class, numbering 36 cases and 
selected in part by means of a composite score 
based on the Police Aptitude Test, the AGCT, 
the Composition Test and the Cornell Word 
Form, recently completed a 12-weeks training 
course. For this group the correlation be- 
tween a weighted score based upon the re- 


gression weights obtained with Class 2 and 
academic grade was .62, or slightly better than 
anticipated. Of the 12 men who were in the 
top third according to this composite score, 
83% had final grades above the median for the 
class, compared with 41% in the middle third 
and only 25% in the lower third. Since in 
Class 3 the validity of the Police Aptitude Test 
was .51 and of the AGCT 55, it would appear 
that the battery as a whole is somewhat more 
predictive in police training than a mental 
alertness test alone“would be. 


Plans for Further Research 


Up to the present time only the first edition 
of the Police Aptitude Test has been used. 
Prior to its administration in connection with 
Classes 1 and 2, none of the material had been 
tried out in any way. An item analysis of this 
test has been completed which is being used as 
a guide in preparing the next edition. As a 
number of the items have been found to be 
unsuitable, it should be possible to improve the 
validity of the instrument. 

Plans for further research include validating 
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the original battery against new criteria as 
they become available, such as service ratings 
after a year and two years of service, remaining 
on the force over a period of years and special 
reports of meritorious service. In the first 
three classes, the experimental tests adminis- 
tered were largely of the non-verbal sort, and 
from that study we can add two or three tests 
to the selection battery. Other types of tests 
are scheduled for experimental try-out from 
time to time, so that the selection battery can 
be modified as further validity information 
becomes available. 


Summary 


1. A battery of psychological selection and 
experimental tests was administered to two 
entering classes (N=72 and N=57, respec- 
tively) of probationary patrolmen at the St. 
Louis Police Academy. A third class (N = 36) 
was used to test the predictive efficacy of the 
battery. Criterion variables were final grade 
in the Police Academy, an achievement test on 
knowledge of police practices, marksmanship 
and ratings by superior officers. 


poowrwet at aS 
Sie peed oo 


2. A specially constructed Police Aptitude 
Test and the Army Genera! Classification Test 
were consistently good predictors of academic 
performance and of the achievement test score. 
At least one non-verbal test, such as the 
Bennett Mechanical Comprehension Test, or 
the Minnesota Paper Form Board, is needed 
as a supplement in order to achieve a multiple 
coefficient of validity in the neighborhood 
of .60. 

3. The best predictors of marksmanship are 
non-verbal aptitude tests, a combination of 
three of which yield a multiple coefficient of 
validity of .33. 

4. None of the tests explored has a signifi- 
cant correlation with rating on the job by 
superior officers. A combination of three non- 
verbal tests and two variables of the Rosen- 
zweig Picture Frustration Study yields a 
multiple coefficient of validity of .29 which is 
significant at the 5% level of confidence. 

5. In the third class (N= 36) a predictive 
score, based upon the regression weights devel- 
oped with Class 2, had a correlation of .62 
with final grade in the Police Academy. 


Received August 3, 1949. 
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Group Differences in Performance on the Meier Art Test 


E. Terry Prothro 


_ University of Tennessee 


and 


Harold T. Perry 


Louisiana State University 


The Meier Art Judgment Test attempts to 
measure an ability which is subject to con- 
siderable development through learning and 
experience. In this paper we have compared 
the scores of several groups of subjects in an 
effort to determine whether membership in 
those groups is associated with differences in 
test performance. If it is, then such infor- 
mation would be of importance to those 
counselors who utilize the test and to those 
students of aesthetics who wish to understand 
the nature and genesis of art judgment. 


Subjects 


A total of 410 subjects were used in this 
study. Of these, 109 were from a state uni- 
versity for whites, 100 were from a state 
university for Negroes, 100 were from the 
tenth and eleventh grades of a city high school 
for whites, and 101 wer¢ from the tenth and 
eleventh grades of a city high school for 
Negroes. All four schools are located in the 
same city in the state of Louisiana. Of the 
total number of subjects, 223 were males and 
187 were females. 


Procedure 


Selection of subjects was by school classes. 
Permission was obtained from the school 
authorities to test al! of the members of a 
given class. Neither authorities nor students 
were given advance information on the exact 
nature of the study. The revised Meier Art 
Judgment Test booklets and forms were placed 
on the desks of all students in the class. 
Each class consisted of approximately 30 
students. 

Because the test is self-administering, few 
verbal instructions were given. It was ex- 
plained that the test was not a part of the 


school curriculum, and that performance on it 
would not affect school standing in any way. 
Only two persons objected to taking the test; 
both were excused. When the subjects filled 
out the personal data sheets, they were specifi- 
cally instructed to indicate the number of 
craftsmen in their ancestry by filling in the 
blanks at the bottom of that sheet. As far as 
could be determined by the experimenters, 
test conditions were about the same for all 
subjects. 


Results 


Examination of the personal data sheets 
revealed that a majority of the subjects had 
received some art training in elementary 
school, some had studied art in high school, 
but very few had had any training beyond that. 

The results are summarized in Table 1. It 
can be seen that there are significant differences 
between mean scores of every pair of groups 
except the male-female groups. The similarity 
of mean scores of the male students and the 
female students was present in each of the 
four schools. In the two Negro schools, the 
males did slightly better; in the two white 
schools, the females did slightly better. None 
of these differences was significant at the five 
percent level. The similarity of the sex groups 
on performance on the Meier Test is especially 
notable in view of the fact that Carroll and 
Eurich! found that women did better than men 
on both the McAdory and the Meier-Seashore 
tests. 

The superiority of university students to 
high school students and of white students to 
Negro students may be attributable to differ- 
ences in the socio-economic status of the 


' Carroll, H. A., and Eurich, A. C. Group differ- 
ences in art judgment. Sch. & Soc., 1931, 34, 204-206. 
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Table 1 


Comparison of Mean Scores on the Meier Art Test 


Group Mean 


96.6 
89.7 
78.3 
74.7 


White University 
White High School 
Negro University 
Negro High School 


University Students 


High School Students 


87.8 
82.1 


White Students 
Negro Students 


93.3 
76.5 


Male Students 
Female Students 


85.6 
84.3 


Students with Craftsmen 
in Ancestry 

Students with No Craftsmen 
in Ancestry 275 


87.2 


84.0 


* t-score is significant at the five per cent level 
** t-score is significant at the one per cent level 


groups. Rodgers* found that better environ- 
ments surrounded children selected as more 
competent artistically. Meier’ has pointed 
out that some aspects of artistic ability are 
subject to considerable development through 
experience, and we believe that the differences 
in daily experiences of members of different 
socio-economic groups may be the basis for 
the differences in their scores. 

Our results confirm Meier’s thesis that there 
is some relationship between “craftsman 
heredity” and aesthetic judgment. The differ- 
ences between persons with craftsmen in their 
ancestry and those with no known craftsmen 
in their ancestry are fairly small, but they are 
significant at the five per cent level. It is 
possible that the ancestry factor is less signifi- 
cant for unselected school populations than for 

2? Rodgers, F. Variation in the aesthetic environ- 
ment of artistic and non-artistic children. Psychol. 
Monogr., 1933, 45, No. 1, 95-107. ' 

* Meier, N. C. Factors in artistic aptitude: final 
summary of a ten-year study of special ability. Psy- 
chol. Monogr., 1939, 51. No. 5, 140-158 


S.D 
3769 S06 
a9 114 7.50** 
s 3 a 34° 
10.5 ° 458 


Diff t 


13.9 


33°" 
128 4.33 


10.5 


11.0 14.98 


13.3 
14.1 ; : oe 


13.9 


populations of art students. Whether the 
relationship is a function of selective recall 
about ancestry, of differences in home environ- 
ment, or of some “innate neuro-physical con- 
stitution” cannot be determined from the in- 
formation now available. 


Summary 

The revised Meier Art Judgment Test was 
given to 410 high school and college students 
in Louisiana. Comparison of mean scores re- 
vealed that the white students did significantly 
better than the Negro students and that college 
students did significantly better than high 
school students. It was suggested that these 
differences may be associated with differences 
in socio-economic status of the several groups. 
Those persons who had craftsmen in their 
ancestry did somewhat better than those who 
did not have craftsmen in their ancestry. 
There were no significant sex differences. 


Received June 10, 1949 
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Use of an Objectivity Key on a Short Industrial 
Personality Questionnaire 


Harold F. Rothe 


Stevenson, Jordan and Harrison, Inc., Chicago, Illinois 


Industrial and personnel psychologists have 
long been concerned with the problem of pos- 
sible “faking’’ by respondents to personality 
questionnaires. Bonnardel (1), in a discussion 
of industrial psychology in France, wrote that 
such questionnaires are little used because it is 
believed that candidates for jobs give the re- 
sponses most favorable to their obtaining the 
jobs. Giese and Christy, in a study reported 
by Tiffin (4), found that the Humm-Wads- 
worth Temperament Scale could be “faked” by 
college students to show higher normal (N) 
scores and less of undesirable characteristics. 
Longstaff (3) has recently shown that the 
Strong Vocational. Interest Blank and the 
Kuder Preference Record could both be faked. 
He suggests that a key like the L or K keys of 
the Minnesota Multiphasic Personality In- 
ventory be incorporated in industrial question- 
naires, not because faking is known to exist, 
but because it might exist. 

The purpose of the present paper is to report 
some experience in the use of a key similar to 
the MMPI L key on a short personality ques- 
tionnaire that has been used in industrial 
practice for the past three years. This form 
has been developed for the use of the members 
of the psychological staff of a management con- 
sulting company and is not available for general 
distribution. Despite this restriction of the 
particular form, the experiences gained with it 
are believed to be helpful to other psychol- 
ogists, and hence this paper is presented. 


Description of the Form 


The form used here consists of fifty items to 
be answered by a YES or NO. It is self- 
administering, has no time limit, and can be 
used for individuals or groups. All items are 
stated in a positive manner and most of them 
refer to behavior rather than to feelings, atti- 
tudes, or interests. The items are scored on a 
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number of keys that are industrially significant. 
One key is the so-called Objectivity key which 
has been patterned after the L scale of the 
MMPI. The other keys are based upon items 
which have been found valid, to some extent, 
on other published keys, although in each in- 
stance the wording has been slightly changed 
to avoid copyright infringement. Several keys 
have only three or four items, and others run as 
high as ten or twelve. For statistical reasons, 
only the longer keys have been analyzed and 
are mentioned in this paper. 

In one very important respect this question- 
naire differs from most standardized person- 
ality questionnaires. This form is primarily 
an interview aid. It is used by trained persons 
in conjunction with a personal history of each 
respondent, various tests, and an interview. 
Being used in this manner, the extreme short- 
ness (50 items), is an advantage rather than a 
handicap. The interviewer can look over each 
statement and each answer, and can interview 
about each item if he so desires. At the same 
time the form can be scored to reveal patterns, 
and the interviewer can also use this pattern 
information as.a part of his interview. 

The longer keys are Emotionality, Social 
Dominance, Drive, Extraversion, and Objec- 
tivity. Since Social Dominance and Extra- 
version are highly correlated with each other 
(in this unrevised form), only the Social 
Dominance key is discussed here. The Emo- 
tionality key has been found to be correlated 
with neuroticism and may be considered as a 
measure of that. The Drive key measures the 
aggressiveness with which the respondent 
strives to reach various goals. It is planned to 
present validation studies fur some of these 
keys in subsequent papers and accordingly 
that problem is not discussed further here. 

Reliability was determined by testing and 
retesting a class of 44 college students with a 
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two-week intervening period.’ The reliability 
coefficients (or better, the consistency coeffi- 
cients) were .84 for the Objectivity, Emotional, 
and Social Dominance keys, and .69 for the 
Drive key. These compare very favorably 
with the test-retest reliabilities of .71 to .83 
reported for the MMPI (2, p. 3). The con- 
sistencies reported in this paper are remarkably 
high considering the small number of items on 
each scale. The high degree of consistency is 
undoubtedly partly a function of the well- 
controlled conditions under which the test- 
retest was made. 

The Objectivity key, as stated before, is 
closely patterned after six of the items on the 
L scale of the MMPI. It would perhaps be 
more accurate to refer to these items as L 
items so that the source of the origination 
would be more clearly recognized and credited. 
The name was changed for practical purposes. 
Since the profile obtained from this form is 
frequently discussed with employers or pro- 
spective employers, it may be an injustice to 
state that a respondent is “lying”’ or “faking.” 
Even though this may be true, insofar as this 
questionnaire is concerned, the fact may well 
be that the respondent is basically an honest 
person. It might be doing him a great in- 
justice to let someone make conclusions about 
the extent to which he “lies” when that con- 
clusion would be based upon six items on a 
questionnaire. 


Relation of Objectivity to Emotionality 


Since the Emotionality key has been found 
to discriminate neurotics from the general 
industrial population with a fairly high degree 
of accuracy, the primary analysis was made 
upon the relation of the Emotionality scores 
to the Objectivity scores. The work of Giese 
and Christy led to the belief that less objective 
persons might show lower Emotionality scores 
than highly objective persons. Conversely, 
highly objective persons might be expected to 
show high Emotionality scores. This assump- 
tion is based upon the assumption that the 
Humm-Wadsworth N score is_ reasonably 
valid. It is also based upon the common- 

! The writer wishes to thank Robert C. McWilliams 


of Northwestern University for conducting this experi- 
ment. 


sense observation that if someone is going to 
fake a questionnaire he will fake items to make 
him look better adjusted or better qualified 
for the job, as he interprets these items. 

A sample of 525 persons was selected. The 
selection was made on the basis of their Ob- 
jectivity scores; 144 persons scoring 0 or 1, 237 
persons scoring 2 or 3, and 144 persons scoring 
4, 5, or 6. The sample was selected from 
master data sheets on which are recorded scores 
of all persons taking this and other tests. 
These persons were all industrial workers in 
various job classifications. For the most part 
they were supervisors, or supervisory candi- 
dates, in both shop and office jobs. Beginning 
with the most recent entry, the first 144 persons 
who scored 0 or 1 in Objectivity were located, 
and their Emotionality scores were plotted. 
In a similar manner the other two groups were 
located and plotted. 

The relation between Emotionality and Ob- 
jectivity could be determined by correlational 
techniques, but have not. This is because of 
the writer’s belief that a table of frequency 
distributions would show the same thing 
graphically and in a more meaningful manner. 
Table 1 contains the distribution of Emotion- 


Table 1 


Distributions of Emotionality Scores Made by 
525 Persons Obtaining Different 
Objectivity Scores 


Number of Persons with Objectivity 
Scores of 


Emotion- 
ality —_—- 
Score Oorl 2or3 4, 5or6 

5 2 0 

17 11 4 

25 20 5 

29 41 17 

24 38 15 

26 37 21 

10 44 20 

2 24 20 
4 18 
2 & & 


2 


_ 
= 
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Mean 
S.D 
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Table 2 


Distributions of Social Dominance Scores Made 
by 525 Persons Obtaining Different 
Objectivity Scores 


Number of Persons with Objectivity 
Scores of 


2or3 


Oorl 


4, 50r6 


0 0 

1 0 

; 1 

19 13 
18 11 
»” 26 
37 


ality scores for these three groups (High, Low, 
and Medium) selected on the basis of their 
Objectivity scores. 

Table 1 shows that persons with high Ob- 
jectivity scores tend to have high Emotionality 
scores, and vice versa. The critical ratio be- 
tween low and medium is 2.4; between medium 
and high is 1.3; and between low and high is 
3.3. The first two of these differences may be 
said to be of little statistical significance. 

It is perhaps more meaningful, from an in- 
terviewer’s point of view, to look at the dis- 
tributions. A respondent who scores 7 on 
the Emotionality key may be considered as 
probably emotional or neurotic if his Ob- 
jectivity score isOor1. But if his Objectivity 
score is 5, that person would be practically 
average in Emotionality, and the interviewer 
would not expect to find symptoms of neuroti- 
cism. The interviewer could profitably spend 
more time interviewing in other areas with this 
highly objective respondent. 

It is possible that the inverse relationship of 
Objectivity to Emotionality that has been 
found here is independent of “faking,”’ i.e., 
that non-objective persons are actually less 
emotional or less neurotic than objective per- 
sons. Data exist, and it is hoped to publish 
them shortly, to rule out this possibility. 


Relation of Objectivity to Social Dominance 
and to Drive 


There is, as shown in Table 1, a relation be- 
tween Objectivity scores and Emotionality 
scores, for this industrial sample. This raises 
the question of the extent to which scores on 
other scales vary with Objectivity scores for 
these persons. Two other long scales of this 
questionnaire have been selected for further 
analysis. 

Table 2 contains the distributions of the 
scores of these same three groups on the Social 
Dominance scale. There is very little differ- 
ence among the Low, Medium, and High Ob- 
jectivity groups in terms of their Social Domi- 
nance. The critical ratios are: .44 between Low 
and Medium; .70 between Medium and High; 
and 1.08 between Low and High. None of 
these differences is significant. 

Table 3 contains the distributions of the 
same three groups on the Drive scale. Again 
the differences between the groups are in- 
significant with critical ratios of .75 between 
Low and Medium; .68 between Medium and 
High; and 1.30 between Low and High. 

The results presented in these three tables 
are based upon an industrial sample composed 
of adults, most of whom have had supervisory 
experience of one kind or another. The sample 
is uncontrolled in terms of intelligence and it is 


Table 3 


Distributions of Drive Scores Made by 525 Persons 
Obtaining Different Objectivity Scores 


Number of Persons with Objectivity 
Scores of 


Oorl 2or3 4,50r6 


0 0 
0 1 
1 1 
6 10 


25 
ol 
ett) 





Use of Objectivity Key on a Short Questionnaire 


assumed that there are no differences among 
the three groups in that respect. It does not 
necessarily follow that similar results would be 
obtained from samples of a different population. 

To be more explicit, it is possible that other 
groups might show a variation in Social Domi- 
nance or Drive scores, depending upon Objec- 
tivity scores. It is reasonable to assume that 
the persons analyzed here, having had super- 
visory experience, might truly rank fairly high 
in both Social Dominance and Drive and hence 
not attempt to “fake” higher scores in those 
respects. It is also possible, however, that 
these persons could not “fake” either of those 
keys, even though they tried to do so. This 
possibility seems less likely than does the 
previous one. 


Summary and Discussion 


Data have been presented to show how the 
Emotionality scores of persons on a short 
industrial personality questionnaire may vary 
with the Objectivity, or L scale, scores of those 
persons. Scores on Social Dominance and 
Drive scales do not vary significantly with 
Objectivity scores. This means that persons 
may “fake’’ a better adjustment than they 
actually have; it also means that the inter- 
viewer can adjust for the faking by inter- 
preting an Emotionality score on norms based 
upon persons of similar Objectivity scores. 

This paper accordingly illustrates a method 
for correcting for faking on short industrial 
personality questionnaires. This is essentially 
the same technique as that of the L scale in the 
MMPI, and the actual items are similar to 
MMPI items (2). There is, however, one 
interesting difference in terms of interpretation. 

One aspect of industrial psychology is the 
testing and interviewing of the key personnel 
within an organization to determine the extent 
to which those persons are adequately qualified 
for their positions. It is reasonable to assume 
that these persons will ‘try their best” on the 
various tests and questionnaires. Indeed, it 
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is the interviewer's responsibility to see that 
they do. Under these conditions it is only to 
be expected that a small degree of “faking” 
or of “stretching the point” will prevail. The 
interviewer expects this and includes it as a 
part of his frame of reference. 

Occasionally, however, an extremely ob- 
jective, or perhaps an extremely naive, person 
is encountered who is frank about himself to 
the point where he penalizes himself on a 
standardized questionnaire. Such a_ person 
might admit that he loses his temper, for 
example, even though he realizes this state- 
ment may be weighed against him. 

Probably the chief use of an Objectivity key 
such as the one described in this paper is to 
protect such persons. That is, the problem 
is not one of locating “‘fakers’ and possibly 
adjusting their results downward. Rather, 
the problem is one of protecting the ‘overly 
honest”” persons and adjusting their scores 
upwards, so that they may be compared favor- 
ably with their more average fellows. An ex- 
ample of this has previously been given: A high 
Emotionality, or neurotic, score may not be 
serious if the respondent is highly objective; 
the same score may be very extreme if the 
respondent is non-objective. The interviewer 
must include the concept of objectivity in his 
interpretation to obtain a correct description of 
the respondent. One method for doing so is to 
maintain different norms for different objec- 
tivity groups, as has been described here. 


Received August 9, 1949 


References 


1. Bonnardel, R. Industrial psychology in France. 
Pers. Psychol., 1949, 2, 47-68. 

. Hathaway, S. R., and McKinley, J.C. Manual for 
the Minnesota Multiphasic Personality Inventory. 
New York: The Psychological Corporation, 1943. 

Longstaff, H. P. Fakability of the Strong Interest 

Blank and the Kuder Preference Record. J. 
appl. Psychol., 1948, 32, 360-369. 

. Tiffin, J. Industrial psychology. New York: Pren- 
tice-Hall, Inc., 1947, rev. ed., pp. 170-171. 


aN I et PMU! BT are lig 


OA aoe hoe oes 


Aa TSAI Me 








The Application of Weber’s Law to Job Evaluation Estimates 


Edward N. Hay 
Edward N. Hay and Associates, Inc., Philadelphia, Pa. 


One method of job evaluation which is gradu- 
ally coming into more general use is the method 
known as factor comparison, devised by 
Eugene J. Benge.' Benge’s method is similar 
to other methods of job evaluation such as the 
predetermined point plan in that the judgments 
are expressed in points, and both methods 
evaluate factors of the job under study rather 
than the job as a whole. The scales used by 
the two methods are quite different. In pre- 
determined point plans certain qualities are 
identified, on the basis of a study of the jobs, 
as necessary for their performance. Scales 
are then constructed and given point values a 
priori to measure these requirements. Factor 
comparison scales, on the other hand, are con- 
structed by dividing the salaries of existing jobs 
among the factors. Nowadays, however, the 
per cent method? of creating factor comparison 
key scales is frequently used. This method 


expresses as percentages, (1) the magnitude of 
each factor of a job in relation to the other 


factors in that job, and (2) the relative magni- 
tude of a factor in one job to the same factor 
in the other key jobs. 

Job evaluation by any method is dependent 
on the judgment of the evaluators. In point 
methods much, if not most, of the judgment is 
in the factors; their selection, definition, point- 
ing, and weighing. In factor comparison the 
judgment is less in these things, and more in 
the comparison of factors of jobs being evalu- 
ated. The evaluators compare one job with 
another in respect of each factor rather than 
match a job against a description on a scale. 
The jobs are familiar to all members of the 
evaluation committee because the original 

! Described in Industrial Relations Magasine, Feb., 
Mar., and Apr., 1932 (now out of print), under the 
title “Gauging the job’s worth.” mechanics of 
the method are also described in Benge, Burke, and 
Hay, Manual of job evaluation. Harper and Bros., 
1941, in chap. IV, pp. 41-56, which is a reprint of an 
article by E. N. Hay, Arranging the right pay, Per 
sonnel J., 1939, 17, 1-7 

* Edward N. Hay. 
scales by the Per Cent Method. 
1948, 32, 456-464 


Creating factor comparison key 
J. appl. Psychol, 
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scale of jobs is created by the committee itself. 
Point methods tend to have steps of irregular 
spacing, while factor comparison evaluation 
scales have intervals that are equal in “sensed 
units of difference.” 

Not much has been written about the theory 
underlying the use of this method. It has been 
found, however, that the judgments expressed 
in factor comparison job rating seem to behave 
in accordance with Weber’s Law.’ By way of 
illustration: an increase in job difficulty at a 
high level calls for a larger salary “raise” than 
at a low level. For example, at the lower end 
of the scale two jobs of just noticeable differ- 
ence in difficulty might have salaries of $100 
and $120 a month respectively. At a higher 
level two jobs of just noticeable difference in 
difficulty might be paid $1000 and $1200 re- 
spectively, but certainly not $1000 and $1020. 
So arithmetic intervals in ‘job difficulty are 
matched by logarithmic increments in salary. 

One of the steps in the factor comparison 
method calls for assigning a percentage value 
to each of the factors in a job (factors which are 
common to all jobs) in such a way that the 
total of all the factor values equals 100%. 
For example, the job of machine bookkeeper 
was selected as a “key job,” and the committee 
made the following assignment of percentage 
values to the five factors as representing in its 
judgment the relative importance of those 
factors in that job: Mental Requirements, 18; 
Skill Requirements, 27; Physical Require- 
ments, 17; Responsibility, 27; and Working 
Conditions, 11. 

The factor scales used by the author are con- 
structed with steps of 15 per cent increment 
and uniform differences in job difficulty. 
Table 1 shows steps for the factor of skill, and 
is representative of other factor scales created 
by evaluation committees. 

Additional jobs that are to be evaluated are 

*See H. E. Garrett. Great experiments in psychology. 
New York: Century Co., 1930, pp. 268-274; and Ed- 


ward N. Hay. Characteristics of factor comparison 
job evaluation. Personnel, 1946, 22, 370-375. 





Weber's Law and Job Evaluation Estimates 


Table 1 


Scale for Evaluating Skill Factor 


Skill Value 
in Points 


Job Title (15% intervals) 


Supervisor of Construction 93 
Credit Analyst 81 
Supervisor Trust Accounting 70 
Head Bookkeeper 61 
Cost Accounting Clerk 53 
Real Estate Maintenance Inspector 47 
Senior Collection Clerk 41 
Junior Branch Teller 36 
Transcribing Machine Operator 31 
Telephone Operator 27 
Stock Clerk 23 
File Clerk 20 
Note Counter 17 
Elevator Operator 15 
Counter Girl 13 
Cleaner 11 
Bus Boy 10 


then compared for their skill requirement with 
the jobs on the scale shown in Table 1. For 
example, the job of Multigraph Operator is 
studied for the skill requirement as outlined 
in the factor definitions. Examination indi- 
cates that the level of skill required for this job 
is most nearly the same as for Telephone 
Operator, and that it requires more skill than 
Stock Clerk and less than Transcribing Ma- 
chine Operator. This three-level comparison 
places the skill requirement at the level of 
Telephone Operator, which gives it 27 points. 
In the same manner the requirement of the 
new job of Multigraph Operator is compared 
with the jobs on the other factor scales and is 
given points accordingly. The sum total of 
the point values for all factors is the point value 
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for the job asa whole. By entering a specially 
constructed table of point and salary values 
we find the salary grade appropriate for the 
new job of Multigraph Operator. 

In factor comparison job evaluation, differ- 
ences in job difficulty at the threshold of dis- 
crimination increase by uniformly equal in- 
crements or steps while their salaries increase 
by a uniform ratio. In working with job 
evaluation data the writer noticed that Weber’s 
Law seemed to apply and decided to experi- 
ment to see if the value of the constant in 


Weber's Law, = C, could be determined. 


R 

As expressed by Wundt, “all judgments are 
governed by the genera! principle of relativity: 
changes are estimated in terms of the thing 
which has been changed.”” Weber put it this 
way: “in comparing objects we perceive not the 
actual difference between them but the ratio 
of this difference to the magnitudes of the two 
objects compared.”” That is to say, the ob- 
served difference between two objects is not 
absolute and independent of the objects them- 
selves, but is relative to their size and is a 
constant fraction of one of them. 

This constant fraction, or ‘‘difference limen,”’ 
must be discovered by experiment, and is 
usually the magnitude at which 75% of the 
judgments agree. Weber found that for 
weights lifted by hand it was 1/30 and for the 
length of lines 1/100. In factor comparison 
job evaluation it is about 1/7 or 15 per cent. 
Data presented here show that ‘‘just noticeable 
differences’ which were correctly discriminated 
by trained job raters about three-fourths of 
the time were 15 per cent apart in salary value 
of the factor intervals. 

In creating the five factor scales in one 
company, by apportioning the values of the 


Table 2 


Agreement Between Job Evaluation Estimates by All Evaluators of a Committee and the 
Final Values they Adopted 


Co. 
1 (1947) 
1 (1946) 
2 (1949) 
2 


(1947) Life Ins. 


Number of: 


Evalu- 
Factors ators 


Judg- 


ments % Agreement 


4095 75 
4005 73 
5400 73 
2544 72 
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Table 3 


Agreement Between Job Evaluation Estimates by All Evaluators of a Committee and the 


Industry 


Bank 
Utility 
Meta! Mfg. 
Life Ins 
Textile 


Jobs 
112 
382 
356 
140 

w@ 


“key jobs” to these five factors, the intervals 
on the scale were adjusted to uniform per- 
centage increments, in accordance with the 
evidence that judgments formed by the use 
of these scales followed Weber's Law. At that 
time it was thought that the different factors 
required different rates of progression. For 
two factors the progression rate was set at 15 
per cent, for another at 11} per cent, for 
another at 20 per cent, and for the fifth at 50 
per cent. After working with these scales for 
a number of years it became evident that the 
first two, using 15 per cent progression, were 
more satisfactory than the others. The last 
two scales mentioned appeared to have too 
coarse intervals, and these scales were re- 
designed at 15 per cent intervals. 
Table 2 shows the results for committees of 
job evaluators in different companies, all of 
whom had had more than one year’s experience 
sing factor comparison job rating. Each 
nember of the committee works independently 
nd submits his ratings in writing. The next 
ay a meeting of all evaluators is held and 
ach member is given a summary sheet showing 
the evaluations submitted by all participants. 
Jifferences are discussed and agreement is 
reached on a satisfactory set of ratings. The 
number of agreements and of differences be- 
tween the ratings by each person and the final 
ratings that are adopted by the committee are 
tallied, which gives percentages of agreement 
for each member. Table 2 shows the average 
percentages of agreement for the members of 
the committees. In the company referred to 
in the first line of Table 2, for example, 91 jobs 
were rated in relation to each of 5 factors by 
each of the 9 members of the evaluation com- 
mittee. This process involved 4095 separate 





Final Values 


they Adopted 


Number of : 
Judg- 
ments 

5.600 
15,280 
14,240 

4,900 

1,500 


Evalu- 
ators 


10 
& 


Factors % Agreement 


66 
62 


Vveeunw os 


8 
7 
5 


judgments. In 75% of these judgments the 
rating of the individual evaluators agreed with 
the final value adopted by the committee. 
Table 2 shows that experienced judges per- 
ceived the same differences in job factors about 
75 per cent of the time. Since these scales 
were constructed with 15 per cent increments, 
these data tend to confirm the finding that 15 
per cent was about the right difference in value 
of the factors. 

Table 3 shows the data from five companies 
in different industries collected during their 
training periods. These data represent the 
judgments of relatively inexperineced com- 
mittees. The percentages of agreement are 
smaller, as might be expected, ranging from 
55 to 66 per cent as against 72 to 75 per cent 
in the cases of the experienced committees 
shown in Table 2. 


Summary 


In factor comparison job evaluation, jobs 
are “rated” for the assignment of salary or 
wage rates by comparing them, one at a time, 
with other jobs with respect to each of several 
factors or elements common to all jobs. 
These factor values constitute “scales,’’ each of 
which consists of an ascending series of values 
representing the relative salary value of that 
factor for a job. This ascending series of 
values progresses by intervals of approxi- 
mately 15 percent. The process of comparing 
one job with another by factors seems to follow 
Weber’s Law, in which the observed differences 
are a constant percentage of one of the job 
factors. 
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Construction and Use of Verbal Analogy Items 
Abraham S. Levine 


Human Resources Research Center, Lackland Air Force Base, San Antonio, Texas 


World War I saw the emergence of the 
verbal analogy item in its present form, when it 
was included as a subtest in the Army Alpha. 
Yerkes (13) credited Thurstone, Otis and 
Bingham for the development of this type of 
item. Since then, the verbal analogy item 
has been incorporated into a host of widely 
used tests including the ACE Psychological 
Examination, National Intelligence Test, Ohio 
State Psychological Test and the 1937 Revised 
Stanford-Binet Test. Although it has been 
commonly used as a subtest, the verbal analogy 
item constitutes the sole type of item in the 
Miller Analogies Test, Form G, which is now 
being widely used as a selection instrument for 
graduate students 

The analogy form of question is usually of 
the recognition type consisting of four or five 
alternative answers, only one of which is 
correct, and the examinee is required to choose 
the correct answer. The following item is 
representative of this form: 

Circle is to square as sphere is fo (circum- 
ference, cube, round, corners, ball). 

Although few test makers who employ ana- 
logies have been concerned with the theoretical 
basis for this item, justification for it may be 
found in the last two of Spearman’s three basic 
laws of cognition dealing with the eduction of 
relations and the use of the educed relation to 
discover the correlate. Spearman (9, p. 180) 
describes the process as follows: “First a pair of 
ideas is given between which a relation has to 
be cognized; and then this relation has to be 
applied to a third idea, so as to generate a 
fourth one called the correlate.”” This is in 
effect what is demanded by a good analogy 
item. 

Several kinds of content have been intro- 
duced into the analogies form. Thus, in addi- 
tion to verbal analogies, there are tests made 
up of spatial or figure analogies and arithmetic 
analogies. Perhaps the most unique departure 
from the traditional type of analogy item was 
put forth in a test discussed by El Koussy (3). 
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He suggested that various data indicated that 
g may best be measured by tests of primary 
perception. In order to eliminate the effect 
of spatial as well as verbal and number factors 
he advocated the use of a new test, Greys 
Analogy Test, which employs varying intensi- 
ties of grey in analogy form. 

However intriguing these other analogy 
ventures may be, the scope of this paper has 
been limited to verbal analogies since the avail- 
able published data on the other forms is as 
yet too meager to permit the adequate formula- 
tion of rules for construction and use. 

The general problem of validity of verbal 
analogy tests may be broken down into three 
aspects in terms of the research literature on 
the problem. First, the question may be 
asked as to how well analogies intercorrelate 
with other verbal tests, such as vocabulary 
and opposites, in order to obtain some estimate 
regarding the saturation of verbal analogies 
with verbal factors that are measured by other 
kinds of items. Perhaps the most important 
aspect of the validity problem is the question of 
intercorrelation of analogies with tests of gen- 
eral ability. In other words, will a test of 
verbal analogies by itself give you a good 
estimate of general academic ability? The 
factorial composition of verbal analogy tests 
may be regarded as still another aspect of the 
validity problem. 

With regard to the problem of intercorrela- 
tions of verbal analogies with other verbal 
tests, there is some apparently contradictory 
evidence. Kelley (6) cites an early research 
by Agnes Rogers using as subjects 61 entering 
high school girls with a mean age of 14.6 and 
an age range of 12-10 to 16-11. Rogers’ data 
showed that the correlation between analogies 
and logical opposites was only +.26. Other 
low or moderate intercorrelations were ob- 
tained by Weisenburg, Roe and McBride (11) 
employing 70 patients from orthopedic and 
surgical wards of three hospitsls in Philadelphia 
carefully selected so as to be a fairly representa- 
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tive sample of the city’s population. All sub- 
jects used English as their native tongue; ages 
ranged from 10 to 59. A correlation of +.52 
was obtained between printed verbal analogy 
tests and the Stanford-Binet Vocabulary sub- 
test, and one of + .31 was reported with Thorn- 
dike word knowledge. Much higher correla- 
tions are reported by both Schneck (8) and 
Anastasi (1) who both used analogies tests on 
college students. Schneck reports a +.68 
with Vocabulary and a +.71 with Opposites. 
Anastasi obtained a +.65 with Vocabulary. 
A plausible interpretation aimed at reconcilia- 
tion of these apparently contradictory results 
would be to regard verbal analogy tests as 
measuring primarily a general ability akin to 
Spearmanian g at the lower ages and LQ. 
levels, and verbal ability at the higher levels. 
Thus it is possible that difficulty is obtained at 
the lower levels largely by relational com- 
plexity, and at the upper levels most of the 
variance is attributable to vocabulary com- 
plexity. This point bears investigation and it 


does not gainsay the possibility of adding com- 
plexity even on the college and graduate 
student levels by the addition of relational 
complexity rather than by abstruse vocabulary 
as has been more often the case. 


Indeed, 
Heim (5) and Hebb (4) independently have 
attempted to develop analogy tests whose 
upper sensitivity would be due to something 
other than vocabulary difficulty. 

To what extent do verbal analogies measure 
what tests of general ability measure? Ter- 
man and Merrill (10) included analogy items 
in the 1937 Revised Stanford-Binet, declaring 
it to be one of the most satisfactory tests since 
it correlated uniformly high (average +.82) at 
each age level with the composite score. In 
the previously mentioned Weisenburg, Roe 
and McBride study (11) the following correla- 
tions were obtained with the printed analogies 
test: Stanford-Binet +.72, Completion Beta 
+ .63, Pintner Nonlanguage +.55, Porteus 
Maze +.62. Such evidence indicates that 
verbal analogies, at least for a heterogeneous 
population, measures largely what more gen- 
eral, even non-verbal, tests measure. 

Whatever evidence there is available on the 
problem of factorial composition of verbal anal- 
ogies points to at least one verbal factor and 
often another factor such as reasoning or deduc- 
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tion. The results of these factor analyses are 
not in complete agreement probably due to dif- 
ferences in the tests and populations used as 
well as methods of analysis and interpretation. 
However, for the most part, the factorial studies 
tend to agree with the logical analysis into 
verbal and reasoning components. 

The reliability coefficients reported for 
analogy tests tend to run somewhat lower than 
for other verbal tests. They characteristically 
bounce around in the .70’s and .80’s as com- 
pared with the .80’s and .90’s obtained for 
such tests as vocabulary, opposites, similarities, 
completion, etc. A combination of some or 
all of the following factors may tend to explain 
this finding as well as point to rules for the 
appropriate use of the analogy item: 

1. Greater improvement through practice is 
reported for analogy-tests than for other verbal 
tests (2). 

2. A greater number of zero scores are re- 
ported for analogy subtests than for other 
subtests, particularly for younger and older 
subjects and subjects with low educational 
attainment, indicating perhaps greater inherent 
difficulty in understanding the analogy ques- 
tion (2, 13). 

3. Analogy subtests tend to be either some- 
what shorter or the time allowance is such that 
the average examinee tends to complete fewer 
items. 

4. The analogy form of question is perhaps 
subject to greater ambiguity and equivocality 
of interpretation than other types of items. 
An important point stressed by Paterson (7), 
and apparently ignored by some analogy item 
makers, is that the first two terms should 
always bear the same relationship to each 
other as the third and fourth term bear. 
Mixed and reversed analogies contribute to the 
ambiguity of the item and convert what would 
otherwise be an exercise in relational thinking 
to a sort of guessing game. 

The discrimination of a verbai analogy item 
may be improved by increasing the degree oi 
association between the wrong alternatives and 
the third term in the item. Zirkle (12) reports 
a study where the distractors in the analogy 
items were subjectively divided into three 
groups on the basis of degree of closeness of 
association with the third word regardless of 
the first two words, and analysis showed that 
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as the distractors went from little association 
to greatest association there was an increased 
tendency to choose them. This tendency was 
greatest for persons scoring in the lowest 27% 
and less for persons scoring in the highest 27%. 

On the basis of the foregoing summary of 
empirical findings and logical considerations, 
the following tentative rules may be formulated 
for the construction and use of verbal analogy 
items: 


1. Sufficient fore-exercises should be in- 
cluded to insure adequate comprehension of 
the analogy form of question. Such a pre- 
cautionary measure will tend to reduce sharply 
the number of zero scores and minimize the 
amount of practice effect. 

2. Items should be arranged in order of 
difficulty with the first few items constituting 
graduated practice in analogy thinking as well 
as serving as shock absorbers. 

3. Avoid mixed and reversed analogies by 
making sure that the first two terms are re- 
lated to each other in the same fashion as are 
the last two terms. 

4. Distractors should be made as plausible 
as possible, and wherever feasible, at least one 
of the distractors should be closely associated 
with the third term in meaning. 

5. The test should include enough items and 
the time limit should be long enough to enable 
examinees to answer sufficient items to insure 
adequate reliability. It is probably desirable 
to use verbal analogy items in a power test 
rather than a speed test since the variable of 
set may play a more important role in under- 
standing what is required than in other types 
of items. 

6. Since it is more difficult to foresee possible 
interpretations of analogy questions, a rigid 
selection on the basis of item analysis data is 
perhaps even more important for analogies 
than for other types of items. 

7. It is perhaps more desirable in measuring 
general ability to attain difficulty by means of 
complexity of relationship rather than through 
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the use of uncommon vocabulary. It should 
be pointed out, however, that the superiority 
of the former type of item as a predictor of 
academic success has not as yet been demon- 
strated. 

8. For optimal effectiveness, the use of the 
analogy item may best be restricted to a 
college or professional school level of young 
adults. Younger and older subjects, as well 
as less intelligent and poorly educated ex- 
aminees, tend to experience the most difficulty 
in understanding the analogy form of question 
and therefore it should be used with great 
caution for these segments of the general 
population. 


Received June 10, 1949. 
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Spread of Vocational Interests and General Adjustment Status * 


Samuel F. Klugman 
Veterans Administration, Philadd phia, Pa. 


Opinions as to the relationship between 
personality inventory scores and vocational 
interest scores vary from one extreme to the 
other. On one hand, for example, there are 
these statements by Tyler and Fowler, re- 
spectively. The former states: “Scares indi- 
cating neurotic tendencies show no appreciable 
relationship to any kind of interest scores” 
(20, p. 67). The latter writes: “General agree- 
ment seems to exist, too, that interest test 
scores are not a dependable basis for conclu- 
sions about a student’s attitudes and adjust- 
ment” (9, p. 26). On the other hand, Darley 
claims, ‘‘Men with primary interest patterns 
‘in technical occupations ... seem to have 
' markedly poorer social adjustments . . . (and) 
| they show a tendency for consistently better 
home, health, and emotional adjustments” (7, 
) p. 470-472). In a similar manner he differ- 
'entiates individuals with Verbal and Social 
Service A patterns from those with Business 
Contact and Social Service B patterns. At a 
later date, based on 80 maladjusted adult 
| cases, he maintains that the data fit “into the 
hypothesis of interests as an outgrowth of 
personality” (8, p. 58). Sarbin and Berdie 
found sufficient relationship to say, “It is 
_ possible in the absence of other interest meas- 
urement to use the Allport-Vernon Scale (a 
personality inventory) to approximate certain 
occupational interest types as measured by 
the Strong test” (15, p. 296). Between these 
two extremes, is this statement by Alteneder: 
“Relationship between vocational interests 
and intelligence and personality traits indicate 
some interesting trends” (2, p. 459). Tussing 
also declares, ‘From this study it does appear 
that a prediction of self-confidence and of 

* Published with the permission of the Chief Medical 
Director, Department of Medicine and Surgery, Vet- 
erans Administration, who assumes no responsibility 
for the opinions nor conclusions by the author. The 
writer wishes to acknowledge his indebtedness to Dr. 
Hans C, Gerdon, Division of Educational Research of 
the Philadelphia School System, for reading this manu 
script and offering several constructive suggestions. 


He assumes, of course, no responsibility for any of the 
writer's statements 


sociability can be made with a fair amount of 
accuracy...” (19, p. 72). However, he 
found no relationship between Strong’s Scales 
and Bell’s Home, Health, and Emotional Ad- 
justment Areas. Strong, too, feels that “in- 
terests as measured today are related some- 
what to certain personality factors and atti- 
tudes” (16, p. 341). 

It is fairly obvious that there is a need for 
further investigation of this problem. Fur- 
thermore, the test which has been used most 
frequently is the Strong Vocational. Interest 
Blank. Perhaps the use of another test would 
vield more consistent results, and toward this 
end the Kuder Preference Record was em- 
ployed in this study. 


Statement of the Problem 


The chief purpose of this study was to 
determine whether any relationship existed 
among veterans, between “‘general adjustment 
status” as measured by the total score of the 
Bell Adjustment Inventory (3, p. 1) and 
“spread of vocational interests’? based on 
Kuder Preference Record profile scores (13). 

So far as this writer is aware, “spread of 
interest’ as used in this investigation is a 
new concept and is not to be confused with 
Berdie’s “range of interests’ which is deter- 
mined merely by counting the number of in- 
terests claimed from a list of 22 recreational! 
items (4, p. 269). Essentially, “spread of 
interest” is similar to Wechsler’s technique for 
determining the significance of a varying score 
by comparing it with the individual’s sub-test 
mean (21, pp. 148-149). In this instance, 
however, it refers to the sum of the deviations 
of all scores from the individual’s mean based 
on the assignment of standard scores (as per 
Bingham, Chapter XIX) to the nine percentile 
scores of the Kuder Preference Record. The 
greater the spread score the more definite are 
one’s likes and dislikes. Stated in another 
way, one who has a profile with only very high 
and very low scores is a person with a more 





Spread of Vocational Interests 


definite pattern of interests than one whose 
scores fall closer to the mean. 

Besides the null hypothesis, there are at 
least two others which may be stated: 


1. The more definite one’s likes and dislikes 
are, resulting in a greater spread of one’s 
vocational interest pattern .(i.e., the greater the 
sum of the deviations from his mean), the 
better adjusted is he. 

2. The more one’s interests are distributed 
in a horizontal manner over a wide area, em- 
bracing many fields (not liking or disliking 
many fields with too much intensity, resulting 
in smaller deviations), the better adjusted is he. 

In addition to the purpose stated above, this 
study seeks to determine whether such factors 
as age, education, and intelligence, as well as 
spread of vocational interests and general ad- 
justment status, are related to the various 
occupational interest areas measured by the 
Kuder Preference Record. 


Procedure 


During the year 1947, by random selection, 
the records of 108 veterans who had applied, or 
had been sent by the Reginal Office, for advise- 
ment to the Veterans Administration-University 
of Pennsylvania Guidance Center, where the 
present writer was serving as Chief, were used 
for this study. These records contained at least 
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the following pertinent information: age, sex, 
formal schooling, race, intelligence quotient (as 
obtained on the California Mental Maturity 
test, advanced form, timed), vocational in- 
terest profile (as per Kuder Preference Record) 
and personality inventory scores (Bell Adjust- 
ment Inventory, adult form). 


The Subjects 

The veterans employed in this study were 
white, native-born males who had a mean age 
of 24.5, S.D. 3.99; had completed 11.3, $.D. 
1.37, grades of schooling; and had an average 
IQ rating of 111.9, S.D. 14.78. This higher 
than average mental ability was due to two 
factors: (1) in order to minimize possible mis- 
understanding of the language elements making 
up the tests only those who had completed at 
least the Eighth Grade were selected; and (2) 
many university students dropped in for ad- 
visement purposes because the guidance center 
was so conveniently located for them. 


Results 


The obtained data for the interest test were 
not essentially different from the norm popula- 
tion. Furthermore, these data were generally 
in agreement with Rose’s veterans who were 
also tested at a university Veterans Admin- 
istration Guidance Center (14). 


Table 1 


Scattergram Showing Relationships Between Kuder Spread-of-Interest Scores and 
Bell General Adjustment Status Scores 


! 


7.01-7.51 1 

6.51-7.00 1 

6.01-6.50 

5.51-6.00 

5.01-5.50 

4.51-5.00 

4.01-4.50 

3.51-4.00 

3.01-3.50 

2.51-3.00 

2.01--2.50 1 1 : 

1.51-2.00 2 1 1 
Total 10 12 26 
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Kuder Spread-of-Interest Scores** 
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* Based on total raw scores 


Bell General Adjustment Status Scores* 
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Table 1 is a scattergram which shows how 
the various scores for General Adjustment 
Status and for spread of vocational interests 
are distributed. By inspection, it may be 
easily noted that the scores are widely scattered 
and do not follow a consistent pattern. This 
is further verified by the calculation of a 
product-moment correlation of 014.07. It 
may be concluded, then, that no relationship 
was found between spread-of-interest scores 
(as measured by our suggested technique) and 
General Adjustment scores (as measured by 
the Bell Adjustment Inventory). 

In addition to the above product-moment 
correlation, others were determined for spread- 
of-interest scores and the three factors of edu- 
cation (i.e. final grade completed), age, and 
IQ rating. For education, a correlation of .31 
was obtained ; for age .29; and for IQ rating .27. 
In all instances, the P.E. was .06. These cor- 
relations are significant at the .01 level and 
each relationship may be described as ‘present 
but slight.” It may be said, then, that the 
veterans who were older, more intelligent, and 
possessed more formal schooling tended to have 
greater spread-of-interest scores than those who 
were younger, less intelligent, and had com- 
pleted fewer years of schooling. 

Table 2 gives a series of relationships be- 
tween five factors and the various Kuder 
interest areas. In dividing the group into 
halves, those with percentile scores above 50 
for a particular interest area were placed in the 
Stronger interest group and those with per- 
centiles below 50 in the Weaker interest group. 
Those with scores of exactly 50 were put in the 
half with the lesser number of cases in order to 
keep the two groups as nearly alike in size as 
possible. The interpretation of critical ratios 
generally follows Guilford (10, p. 61). For 
those interested in interpretations based on 
Student’s distribution for a group as large as 
this, a C.R. of 1.98 indicates confidence at the 
5 per cent level; 2.36 at the 2 per cent level; 
and 2.63 at the 1 per cent level. In terms of 
factors studied, these results were obtained: 


1. In no area did spread of vocational in- 
terest scores yield differences beyond reason- 
able doubt between those with stronger and 
weaker vocational interests. 

2. Reliably better adjustment scores were 
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obtained by those with stronger Scientific in- 
terest scores. Those with weaker Artistic 
interest scores earned scores which indicated 
fair certainty of difference in their favor. Had 
the Social Service interest scores yielded a C.R. 
of 1.98 instead of 1.92, it would have been 
significant at the 5 per cent level of confidence 
in favor of those in the weaker half. It is 
interesting to note, in passing, that Kimber 
had: found that Bible students had “less free- 
dom from nervous symptoms than the average 
person, the prevailing interest was in social 
service, and there was a noticeable lack of 
interest in computational and clerical activi- 
ties” (11, p. 233). It will be seen that the 
present findings are in agreement because those 
with stronger Social Service, weaker Computa- 
tional, and weaker Clerical interests had poorer 
adjustment scores. However, none of the dif- 
ferences is significant although, as pointed out 
above, there is a relatively high level of con- 
fidence for the Social Service area. 

3. Only in the Mechanical area was there 
obtained a difference large enough to point to a 
relationship with IQ ratings. With fair cer- 
tainty, it was found that those with stronger 
Mechanical interests earned lower IQ ratings. 
An interesting result may be noted with ref- 
erence to the Scientific category; namely, IQ’s 
differentiated least between those who liked 
and those who disliked scientific activities as 
stated in the Kuder test. This is contrary 
to a conclusion drawn by Super (17, p. 23) 
from Strong’s findings (16). On the other 
hand, it supports his statement concerning 
Social Service interests (p. 24). 

These findings are essentially in agreement 
with those of Adkins and Kuder who, using a 
test of intelligence, stated that “. . . the inter- 
pretation of preference scores as indicative of 
the presence or absence of special abilities is 
unwarranted by the results of this investiga- 
tion” (1, p. 261). Similarly, Strong writes that 
“the differences are too slight to be usable in 
prediction of individual cases” (16, p. 334). 
Triggs’ highest coefficient of correlation be- 
tween Kuder interest scores and ACE total 
scores was only .27 (Literary category), once 
again indicating low relationship (18, p. 349). 

4. Age apparently bore no relationship to 
any of the interest areas, since all critical ratios 
were even below the lower criterion of 2.00. 
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5. Education seemed to be significantly re- 
lated to more interest areas than any other 
factor investigated in this study. Those with 
weaker Mechanical, weaker Clerical, and 
stronger Computational interest scores had 
significantly completed more years of formal 
schooling. In this connection, Crosby ob- 
tained results that “show a positive relation- 
ship between interests in certain scales of the 
Preference Record and achievement in some 
school subjects” (6, p. 103). Triggs also 
found relationships which “tend to indicate 
that interests and achievement are not totally 
unrelated” (18, p. 354). The present writer, 
on an earlier occasion, when he investigated 
the effect of schooling on clerical interests, 
stated, “. . . it is not maturation which is re- 
sponsible for the improvement in scores . . . 
but probably schooling” (12, p. 95). 

The data in Table 3 are based on the findings 
reported in Table 2. Whereas, in the earlier 
table the relationships of the various factors 
are noted within each of the interest categories 
(i.e., those with stronger are compared with 
those having weaker interests), the present 
table is concerned with the relationships of 
these factors among the interest categories. 


Klugman 


Furthermore, the data in this table deal with 
positive interest alone in the sense that com- 
parisons were made only between the stronger 
halves (above 50th percentile) of the interest 
categories. Finally, in Table 3 are the signifi- 
cant differences only, i.e., those critical ratios 
that are significant at least at the 5 per cent 
level of confidence. The sole exception is so 
close to the criterion of 1.98 that it is not amiss 
to include it in this table. 

One important fact to note in this table is 
that the Mechanical interest category appears 
to be significantly related to all factors. Those 
with stronger Mechanical interests are less 
definite in their likes and dislikes of other 
vocational interests (as measured by the sug- 
gested technique) than are those with stronger 
interests in the Clerical, Computational, Scien- 
tific, Social Service, Musical, and Literary 
areas. General Adjustment status appears to 
be poorer for them (since lower inventory 
scores are more desirable) than for those with 
strong Scientific interests. IQ ratings were 
lower for them, too, than for those with 
stronger interests in the Computational cate- 
gory. In addition, those with stronger Me- 
chanical interests were reliably older than those 





Table 3 


Significant Inter-Category Difference Means for the Various Fav:tors 


Factor: 


Spread of Interests 


General Adjustment Status 


IQ Ratings 


Age 


Education 


* Higher mean found in second interest category of each pair 


Inter-Category 


Interest Categories Difference* 


Mechanical and Clerical 75 
Mechanical and Computational 69 
Mechanical and Scientific 67 
Mechanical and Social Service 62 
Mechanica! and Musical 
Mechanical and Literary 


Scientific and Artistic 
Scientific and Mechanical 
Scientific and Social Service 
Scientific and Musical 
Computational and Artistic 


Mechanical and Computational 


Literary and Mechanical 
Persuasive and Mechanical 


Mechanical and Computational 
Mechanical and Literary 
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Spread of Vocational Interests 


with positive Literary and Persuasive interests. 
They also had less formal schooling than those 
with stronger Computational and Literary 
interests. 

Those with stronger interests in the Scientific 
area were better adjusted, to a fair degree of 
certainty, than those with interests in Artistic, 
Social Service, Mechanical, and Musical fields. 
Similarly, those with positive Computational 
interests were emotionally more stable than 
those with Artistic ones. 

It is interesting to note that Education, 
which was the factor related to most intra- 
category differences, did not play so important 
a role in inter-category differences. 


Summary 


The records of 108 veterans who had applied 
for advisement at the Veterans Administration- 
University of Pennsylvania Guidance Center 
were selected at random in order to determine 
what, if any, relationship existed between 
vocational interest and personality inventory 
scores as measured by the Kuder Preference 
Record and the Bell Adjustment Inventory, 
respectively. Of special interest was the de- 
termination of the value of a new measure of 
spread of vocational interests based on mean 
deviations of an individual’s standard scores. 
In addition, the factors of age, IQ ratings, and 
education were studied in relation to interests. 

The following findings were obtained: 


1. Spread of vocational interest, as measured 
in this study, showed no relationship with 
-General Adjustment Status. This lack of 
relationship was also found in a comparison 
between those with stronger and those with 
weaker interests in the various areas. 

2. There was a slight tendency for the older, 
more educated, and more intelligent veterans 
to have greater spread-of-interest scores than 
the younger, less educated, and less intel- 
ligent ones. 

3. Spread-of-interest scores succeeded in dif- 
ferentiating between those with stronger Me- 
chanical interests and stronger Clerical, Com- 
putational, Scientific, Social Service, Musical, 
and Literary to a degree of fair certainty at 
least. In all instances, the mean spread score 
of stronger Mechanical interests was less than 
the mean spread score of the others. 
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4. Veterans with stronger Scientific interest 
scores earned reliably better adjustment scores, 
as did those with weaker Artistic interest 
scores (but to a lesser degree), than their 
opposite halves. General Adjustment Status 
for those with stronger Scientific interests was 
significantly better than for those with stronger 
Artistic, Social Service, Mechanical, and 
Musica! interests. To a lesser extent, this 
is also true for those with positive Computa- 
tional interests over positive Artistic ones. 

5. IQ ratings did not differentiate within 
any of the interest categories except for the 
Mechanical area where those with weaker 
scores earned higher IQ’s to a degree of fair 
reliability. A significant difference (at the 5 
per cent level of confidence) was obtained, on 
the basis of these scores, between those with 
positive Mechanical and Computational in- 
terests. 

6. The factor of age did not succeed in dif- 
ferentiating any of the veterans on the basis of 
stronger versus weaker vocational interest 
scores. Significant age differences (at the 5 
per cent level) were found between positive 
Mechanical and positive Literary and Persua- 
sive interest areas respectively. Those with 
stronger Mechanical interests were older than 
those with high interests in the other two fields. 

7. Education, more than any of the factors 
studied, was related to a significant degree 
when studied on an intra-category basis. How- 
ever, this factor was not so effective in differ- 
entiating among those with positive interests. 

8. The Persuasive, Literary, and Musical 
areas of interest were found not to be related 
with certainty to any of the factors investigated 
when studied on an intra-category basis. At- 
tention is called to the Social Service area which 
is barely outside the 5 per cent level of con- 
fidence in relation to General Adjustment 
Status. When investigated on the basis of 
positive inter-category interests, however, all 
were found to be related to one or more of 
the factors. 

9. The Mechanical interest area was related 
to all the factors on an inter-category basis with 
at least fair certainty. On an intra-categary 
basis, the same relationship was found for all 
factors except spread-of-interest scores. 

10. Relationship between a personality in- 
ventory total score (Bell Adjustment) and 
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vocational interest test scores does not appear 
to be greater when the Kuder Preference 
Record is used than when the Strong Voca- 
tional Interest Blank is employed. To use 
Strong’s descriptive statement, the scores are 
“related somewhat.” Therefore, it does not 
appear to be feasible, at this time, for voca- 
tional counselors, clinical psychologists, and 
others who make use of these types of tests, 
to infer the presence of definite relationships 
except in a limited manner. 

Received June 20, 1949. 
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Kuder Interest Patterns of Psychologists * 


Malcolm L. Baas 


Purdue University 


The primary purpose of this study was to 
secure information concerning motivations, in- 
terests, or preferences of professional psychol- 
ogists, and of graduate students in psychology. 
Significance of this purpose is reflected in such 
studies as The Research Project on the Selection 
of Clinical Psychologists (6) being conducted at 
the University of Michigan under the super- 
vision of E. Lowell Kelly and Donald W. 
Fiske. 

Cognizance of such information may in- 
troduce considerations in the guidance of 
students toward or away from psychology; 
and also considerations in the evaluation of 
personality patterns among practicing profes- 
sional psychologists. 


Procedure 


Groups Used. Subjects from four divisions 
of the American Psychological Association 
were selected for this study. These divisions 
were: Clinical, Industrial, Counseling and 
Guidance, and Experimental and Theoretical. 
From each division, sixty fellows were ran- 
domly selected and subsequently contacted. 
Also participating in the study were groups 
of graduate students in both Industrial and 
Clinical Psychology at Purdue University. 
The graduate students were candidates for 
either the M.S. or the Ph.D., degree at the 
time of this study. 

Administration of the Kuder Preference 
Record. The subjects were requested to com- 
plete the Kuder Preference Record, Form BM, 
and return it to the author. From the origi- 
nally selected professional group of 260, returns 
permitted a statistical analysis of 29 Industrial, 
27 Clinical, 26 Counseling and Guidance, and 
29 Experimental and Theoretical Psycho- 

* This article is based upon the author’s thesis, 
entitled “A Study of Interest Patterns Among Pro- 
fessional Psychologists,” submitted to the Faculty of 
Purdue University in partial fulfillment of the require- 
ments for the degree of Master of Science, February, 


1949. The thesis was directed by Professor Franklin J. 
Shaw. 
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logists, a total of 111. This is a 46 per cent 
return. None of the psychologists considered 
in this final group was a fellow in more than 
one of the divisions as listed. From the 
student groups, 21 Clinical and 25 Industrial 
Psychology Graduate Students were con- 
sidered in the final study. 

Treatment of Data. Mean raw scores’ on 
each of the nine Kuder Scales were obtained 
for each of the Professional groups, and for 
the graduate student groups. Similarly, per- 
centiles, standard deviations, and standard 
errors were obtained. 

Using the Kuder Adult Norms for Men, 
percentile ranks for the mean raw scores were 
found. The mean scores were also interpreted 
in terms of norms derived from the entir~ pro- 
fessional psychologist sample. 

To indicate significance of mean-differences 
between the divisions, Student’s /-ratios were 
tabulated. These ratios serve to test the null 
hypothesis concerning the differences in means 
between professional groups, and between 
student and professional groups. 

All statistical data have been reported in 
tabular and profile forms, but because of their 
extensive length they must be deleted from 
this paper 

Results 


For the entire professional sample (Table 1) 
two scales exhibited strength of interest. The 
Scientific and Literary scores were above the 
75th percentile of Kuder’s Adult norms for men. 

The four divisions seem fairly consistent in 
reflecting the scientific and literary interests. 
For these two scales, only one significant 
difference in means occurred. The Scientific 
Scale difference between the means for the 
Theoretical and the Clinical Divisions was 
significant at the 2% level of confidence. In- 
asmuch as the difference favored the Theo- 

' Because of the extensive amount of statistical data, 
the tabulations and profile presentations have been 


deleted from this article. The origina! thesis is filed 
in the Purdue University Library 
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Table 1 


Group Mech. Comp. Sci. 


Psychologists in general 32 67 
Clinical 32 55 
Industrial 40 75 
Counseling 23 4 
Theoretical 35 70 
Student Clinical x 22 
Student Industrial 26 70 


retical Division, this outcome seems consistent 
with the divergent patterns of activity of the 
two fields. 

Low interest areas for the professional psy- 
chologists were reflected in the mean scores for 
the Mechanical, the Persuasive, and the Cleri- 
cal Scales. Significant differences in means 
were found on the Mechanical Scale, the 
Counseling Division’s mean score being sig- 
nificantly less than the mean score for either 
the Theoretical or the Industrial Divisions. 
Levels of confidence for the differences were 
5% and 1%, respectively. The Industrial and 
Counseling Divisions had higher mean scores 
on the Persuasive Scale than the other divi- 
sions. Industrial Psychologists evidenced a 
stronger interest on this scale than either the 
Clinical or the Theoretical Divisions, with 
significance at the 1% level of confidence. On 
the Clerical Scale, the difference between the 
means for the Industrial and Clinical Divisions 


Scales 


Art. 
sy 
&7 
40 
30 
67 
46 
12 


was significant at the 5% level of confidence, 
favoring the Industrial group. 

The middle-range mean scores also showed 
significant differences between certain of the 
tested groups. The Artistic Scale differen- 
tiated between the Counseling and Industrial 
divisions, favoring the Industrial Division at 
the 5% level of confidence. The same scale 
differentiated between the Theoretical Division 
and the Counseling Division. For both these 
mean-difference comparisons, the Theoretical 
Division mean score was significantly higher 
at the 2% level of confidence. No significant 
differences occurred for the Computational 
Scale mean-score comparisons, but the Social 
Service Scale exhibited significant mean dif- 
ferences in all the inter-division comparisons. 
For the Social Service Scale, mean-differences 
at the 1% level of confidence favor the Clinical 
Division over both Industrial and Theoretical 
Divisions; and the Counseling Division over 


Table 2 


Profile Statistics for 111 Professional Psychologists 


Mean 
Raw 


Scale Score 


Mechanical 
Computational 
Scientific 
Persuasive 
Artistic 
Literary 


68.9 32 
40.0 67 
77.8 82 
57.7 24 
48.2 59 
O48 86 
17.4 60 
78.2 60 
44.7 30 


Musical 
Social Service 
Clerical 


%-ile M 
(Kuder 
Norms) 


%-ile M 
(Sample 
Norms) 


Standard 
Deviation 


50 17.5 
50 10.1 
50 12.7 
50 17.6 
50 16.2 
12.1 
8.8 
22.0 
12.6 
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both Industrial and Theoretical Divisions. 
The Social Service mean-difference between 
the Industrial and Theoretical Divisions favors 
the Industrial Division at the 2% level of con- 
fidence. And the Counseling Division’s Social 
Service mean score is significantly higher than 
the Clinical Division’s mean score at the 5% 
level of confidence. 

Significant differences also prevailed be- 
tween the mean scores of the professional 
psychologists and their student counterparts. 
Only the Literary and the Clerical Scales failed 
to differentiate between the Professional Clin- 
ical Psychologists and the Clinical Psychology 
Graduate Students. With regard to the Pro- 
fessional Industrial Psychologists and the 
students of Industrial Psychology, four scales 
failed to show differences in means. The four 
scales were: The Mechanical, The Computa- 
tional, The Musical and The Clerical Scales. 
In view of these results it seems possible that 
preference patterns are considerably affected 
by professional experience, although Strong’s 
work (9) would not lend support to this view. 
It is possible that interest patterns will become 
more stable and that strong interest areas will 
become better established as experience con- 
tributes toan individual’s preferences. Several 
investigators (1, 2, 4, 7, 11) have indicated, 
nevertheless, that the Auder Preference Record 
can be of help ia selecting and guiding students. 


Summary 


In this investigation an effort was made to 
identify characteristic interest patterns for 
four different groups of professional psycholo- 
gists and for two groups of student psychologists. 
Employing the Kuder Preference Record, pref- 
erences of groups selected from four American 
Psychological Association divisions and two 
groups of psychology graduate students were 
obtained. 

Having received the completed Kuder Pref- 
erence Record forms from sample groups of 
fellows in the Clinical, the Industrial, the 
Counseling and Guidance, and the Theoretical 
and Experimental Divisions, a_ statistical 
analysis of these results was made. A similar 
analysis was performed on the graduate student 
data. 
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The findings revealed high scores for all pro- 
fessional psychologists on the Scientific and 
Literary Scales of the Kuder Preference Record. 
Weak interests were evident on the Me- 
chanical, the Persuasive, and the Clerical 
Scales. Several significant differences be- 
tween the groups studied were found on the 
various scales. 

The findings suggest that professional ex- 
perience may contribute to preference scores, 
although Strong’s work on age changes in in- 
terest patterns would not support this. How- 
ever, this particular investigation has not in- 
corporated an examination of subjects’ early 
interest backgrounds. Age differences, ex- 
perience differences, and environmental in- 
fluences should be considered. And the small 
samples, as well as having used only Purdue 
University graduate students, are also limit- 
ing factors. 


Received July 5, 1949. 
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A Study of Shooting Glasses by Means of Firing Accuracy * 


Sherman Ross 
Bucknell University, Lewisburg, Pennsylvania 


The use of “shooting glasses” is apparently 
a very old idea in the field of hunting or of 
range firing. In the museum at Fort Necessity, 
a pair of old-fashioned spectacles with red glass, 
labelled: “Hunting Glasses, about 1830” are 
to be found. 

There have appeared on the commercial 
market several different types of glasses, with 
an assortment of filters, which have been 
declared to be shooting glasses. The major 
purpose of these glasses is to provide better 
resolution of the target, under conditions of 
glare, haze, etc. The glasses have been pro- 
duced commercially under such trade names as 
“Rayban,” “Calobar,” and “Eaglesight.” 

The use of filters in ‘‘cutting haze” and in 
providing better resolution of targets is a well- 
established notion in the field of vision. Ina 
recent report, Verplanck' has studied the 
effectiveness of selected neutral and colored 


filters in extending the visual range at which 
neutral targets, viewed against a sky back- 
ground, may be discriminated in the presence 


of haze. The results of the experiments indi- 
cate that the use of colored or neutral filters 
did not increase the range at which neutral 
targets, viewed against a sky background, may 
be discriminated. The present experiment 
deals not with the problem of discrimination, 
but with the complex of visual-psycho-motor 
variables represented by the actual firing of a 
rifle at a target. The purpose of the present 
study was to determine the effectiveness of 


* This study was performed at the Medical Field 
Research Laboratory, Camp Lejeune, North Carolina, 
during the fall of 1944, with the cooperation of the 
Range Battalion, Camp Lejeune. Opinions expressed 
in this report are those of the author and are not to be 
construed as official or as representing the views of the 
Navy Department or of the Naval Service at large. 
The writer should like to express his thanks to S. B 
Williams, College of William and Mary, and to S. B 
Lyerly, University of North Carolina, for their help on 
the manuscript. 

' Verplanck, W. S. A field test of the use of filters 
in penetrating haze. Report on Bu M & S Research 
Project HM-011-003, dated 6 June 1947. From the 
Medical Research Department, U. S. Submarine Base, 
New London, Conn. Pp. 11. 
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several types of plastic filters when used as 
shooting glasses by a group of skilled Marine 
Corps riflemen during range firing. 


Method and Procedure 


It will be convenient to discuss the method 
and procedure under the following headings: 
(1) Subjects, (2) Filters, (3) Firing Schedule, 
and (4) Method of Scoring. 


1. Subjects: The subjects for the experiment 
were a group of 21 riflemen with the highest 
qualifications, selected from a group of 28 range 
coaches who had reported for duty during the 
previous week. 

2. Filters: Six types of filters and one clear 
control were used. The characteristics of 
these filters are shown in Table 1. This table 
shows the Polaroid Corporation code numbers, 
the polarization properties of the filter, the 
color, and the transmission characteristics. 
The plastic filters were placed in identical, 
rubber mounted frames (Polaroid 1065), and 
were marked with identifying letters. 

3. Firing Schedule: The tests ran for three 
consecutive days. Firing was begun between 
0815 and 0845 and ended between 1100 and 
1130. The weather was clear and sunny, and 
there were relatively few clouds during the 
tests. The sun came from over the right 
shoulders of the men on the firing line. The 
subjects were divided into three groups of 
seven men each. This was done for conven- 
ience in handling the goggles and in scoring 
the targets. Sighting in of the rifles was done 
each day previous to firing. Eight shots, slow 
fire, prone at 200 yards with a standard target, 
constituted a string. The M-1 rifle was used 
throughout the tests. The daily plan of the 
firing for each group of seven men is presented 
in Table 2. As will be seen, the firing was done 
in alternate strings: with goggles and without 
goggles. However, when goggles were used, 
each man used a different type. After Relay 1 
was fired by the first group of seven men, the 
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Table 1 


Showing the Characteristics of the Filters 





Polaroid 


Corp. No. 
xCo2 


Polarization Color 


Filter 
Non- Polarizing 


XN10 
HN32 
MG30 
XY68! 
XG3P 
X0O50* 


Neutral 
Neutral 
Neutral 
Amber 
Green 
Light Red 


Non- Polarizing 
Polarizing 
Polarizing 
Non- Polarizing 
Non- Polarizing 
Non- Polarizing 


' This filter selected because of its resemblance to the commercially produced “Rifleite” shooting glass. 


Neutral 


Transmission Characteristics 
Essentially complete transmission throughout 
visible region except for reflection losses 
Approximately 10% overall visual transmission 
Approximately 32% overall visual transmission 
Approximately 30% overall visual transmission 
Approximately 68% overall visual transmission 
Approximately 30% overall visual transmission 
Approximately 50% overall visual transmission 


? This filter selected due to its resemblance to the commercially produced “Rayban” and “Calobar” glasses. 
* This filter selected since it resembled the commercially produced “Eaglesight” shooting glass. 


second group of seven men fired Relay 1. 
After Relay 1 was fired by the third group of 
seven subjects, the first group fired Relay 2, 
etc. The firing for a single day for each sub- 
ject therefore was composed of 14 strings of 
eight shots, seven strings without goggles, and 
one string with each pair of goggles. 

4. Method of Scoring: The groups were 
plotted immediately by spotters in the pits on 


reduced score sheets (1 inch equal to 1/16 
inch). Each hit was plotted exact'y in ac- 
cordance with the location of the hit on the 
target by the score keeper. These score sheets 
provided the basic data for the analysis of 
the results. Not more than 5 per cent error 
inheres in this method of scoring. 

After each day’s firing, the data were ana- 
lyzed according to the standard method known 


Table 2 
Firing Schedule 





Subject 
Al A2 


None* 
Ase 


A3 A4 AS A6 Al 


None None None None None None 
_ a Cc D E. Ik G 


None None None None None None None 
K & D E F G \ 


None None None None None None None 
Cc D E F G \ B 


None None None None None None None 
D E F G \ Bb * 


None None None None None None None 
E F G A 13) S D 


11 None None None None None None None 
12 F G A B ec D E 


13 None None None None None None None 
14 G A B Cc D E F 


* Without goggles. 
** Goggle “A,” goggle “B,” 


etc. 
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as the Mean Radius Method. In this method 
the Group Center is located by a simple — 
process. The distance (linear in all cases) 
between the determined Group Center and the 
Center of the target is known as the Group 
Off Center measure. When the Group Center 
has been determined, the deviation in inches 
of each shot from this point is measured. The 
average deviation is known as the Mean 
Radius, and indicates the variability about this 
center. We than have a measure of central 
tendency (Group Center) and a measure of 
variability about this center. In the experi- 
ment, the best measure of firing accuracy is 
given by the Mean Radius score. The Group 
Off Center measure is subject to influence by 
constant errors such as improper sighting, wind 
shifts, etc. 

The Method of Mean Radius is a method 
used to obtain a precise measure of the 
variability of a group of shots. It is based 
upon some simple statistical concepts. The 
method as used by riflemen is as follows: 

Given the distribution of shots, two coordinates are 
drawn as follows: (1) a horizontal coordinate through 
the lowest shot in the group, and (2) a vertical coordinate 
through the shot nearest the left hand side of the target. 
From the horizontal coordinate, the distances are 
measured (in inches) between the coordinate (lowest 
shot) and each other shot. These distances are aver- 
aged, and a new horizontal coordinate drawn on the 
target as a dotted line. The same operation is per- 
formed with the vertical coordinate. When the new 
vertical (dotted) line is drawn in, the point of the inter- 
section of the two dotted coordinates is known as the 
“Group Center,” 

From this Group Center, the shortest linear distance 
(in inches) between it and each of the shots is measured 
and recorded. The average distance is determined for 
each distribution of shots fired and is called the “Mean 
Radius.” 


It can be readily seen that this is a quick 
method of determining a measure of central 
tendency (the Group Center) and a measure of 
dispersion around that center (The Mean 
Radius). 


Results 

Basic Data: The basic data upon which all 
the calculations were made are not shown in 
this report. The Mean Radius score (in 
inches) for each string of shots under each 
testing condition for every subject was cal- 
culated for each day of firing. An inspection 
of Table 3 reveals that the average Mean 
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Table 3 
Summary Table Showing Average Mean Radius Scores 


Without “oe sie kaa aes eee 
Goggle A ‘ 4.5 
Goggle B 4.3 
Goggle C 4.2 
Goggle D 
Goggle E 
Goggle F . 

Goggle G..... 


Radius, regardless of test condition, varied 
from 3.9 to 4.9 inches, and the standard devia- 
tion varied from 0.8 to 1.4. However, a study 
of the variability of the average Mean Radii 
does not indicate any significant trend. 

Summary of Data on Firing Accuracy: The 
averages of the Mean Radius scores for each 
test condition (without goggles and with each 
type of goggle) summarized for al! days of 
firing are presented in Table 3. The evidence 
suggests the conclusion that the wearing of 
any goggle tends to reduce firing accuracy. 
Except for goggles “E” and “‘C”’, the “without 
goggle’ control firing was superior to firing 
with goggles. Hence, the control goggle ‘‘A”’ 
(clear plastic) is probably the best standard 
with which to compare the effect of other 
filters, rather than the “without goggle” condi- 
tion. Whether or not the small differences 
obtained can be regarded as significant will be 
discussed in the following section. 

Significance of Obtained Differences Among 
Test Goggles: In order to estimate the signifi- 
cance of the differences between shooting with 
various goggles and shooting without goggles, 
the following statistical test was made. The 
largest average Mean Radius score difference 
was taken from Table 3. This difference rep- 
resented a magnitude of 0.4 inches in the 
average Mean Radius between goggle “E” 
(4.1) and goggle “A”’ (4.5). The Critical Ratio 
of this difference was found to be 1.85, or 96.5 
chances in 100 that the obtained difference was 
a real one and not due to chance. This level 
of significance is below the one usually accepted 
in work of this kind. Thus, we have shown 
that even the largest of Mean Radius differ- 
ences, though suggestive, is not statistically 
significant. It was therefore not considered 
necessary to investigate the other differences 
along these lines. 
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Daily Trends: In Figure 1 is shown the dis- 
tribution of Mean Radius scores (without 
goggles) for the first, second, and third days of 
firing. In Figure 2 is shown the distribution 
of Mean Radius scores with goggles for the 
same three days of fire. It will be noticed 
that there is no significant day-to-day trend in 
either of the two figures, and that the distribu- 
tions may be superimposed. It may therefore 
be concluded that the continued wearing of 
the various shooting ‘goggles did not in any way 
significantly increase the marksmanship score. 


| 


‘ 
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Analysis of Variance: An analysis of variance 
was made, the results of which are summarized 
in Table 4. The following interpretation is 
offered. The differences among the goggles 
are clearly non-significant, and there is no 
evidence of day-to-day general shifts in firing 
accuracy. 

The differences among the subjects are highly 
significant which is, of course, not surprising. 
There is no statistical significance in the goggles 
X subjects interaction. In other words there 
is no evidence that, for example, goggle A is 


O———CO First pay 





7 7 7 as as J 
65 70 7% 66 85 90 $ WO 
MEAN RADIUS SCORE 


Daily mean radius score of 21 subjects firing without goggles. 
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e@——® scorn my 
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Fic. 2. 


Daily mean radius score of 21 subjects firing with goggles. 
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Table 4 
Analysis of Variance 














Sums of 
Squares 


Source of Variance 


1. Gogpes “sa 


. Days 1.2456 


. Subjects 154.8697 


. Goggles X Subjects 128.2571 


. Days X Subjects 65.2087 


. Goggles X Days 33.3077 


. Error 224.7513 


. Total 


613.0183 


better than goggle B for subject 1, but worse 
for subject 2, etc. The days X subjects inter- 
action is highly significant and means merely 
that the relative standings of the subjects in 
firing performance is a function of the day on 
which performance is measured. In other 
words, subject 1 may be better than subject 
2 on one day, but poorer on another day” 

The value for the goggles X days interaction 
is highly significant and poses a real problem. 
It must be recalled that the two simple vari- 
ances, goggles and days were not significant. 
This may mean that one goggle is significantly 
better than another on one day, but not on 
another day. Over the period of days, the 
differences could cancel out. 

Several factors might have varied during 
the test situation. For example, the condi- 
tions of light, haze, etc., might have varied 
slightly from day-to-day in ways which would 
affect the relative efficiency of the several 
goggles, 

There may have been differences in the sub- 
jects which could have come into play. The 
preferences of the subjects might have in- 

* This is also to be expected, although it is a prin- 
ciple which athletic coaches are more familiar with (and 
—s deal with more effectively) than experimental 


yehologists. There may be changes in general fitness 
rom day to day, level of aspiration, etc. 


(non. sig.) ; 
F< 10 


(non. sig.) 
F< 10 


8.27 
(1% level) 


1.14 
(non. sig.) 


4 1,74 


(1% level) 


2.97 
(1% level) 


12 


240 
4H) 


fluenced the day’s firing. The subjects were 
interviewed in regard to their preferences, but 
the stated preferences bore no particular re- 
lationship to performance. 


Summary 

Six different types of plastic filters were 
tested as shooting glasses at the Rifle Range, 
Camp Lejeune, North Carolina. A neutral, 
clear plastic was used as a control. The sub- 
jects were 21 riflemen with the highest qualifi- 
cations. They fired the standard M-1 rifle 
from the prone position at an ‘‘A”’ target from 
a distance of 200 yards. 

The shots were plotted immediately on re- 
duced score sheets of target design. The 
accuracy of fire was determined by the stand- 
ard method, known as the Mean Radius 
method. 

No statistically significant difference in per- 
formance, as determined by the Mean Radius 
scores, was found between firing with glasses 
and firing without glasses. No significant day- 
to-day trends were found. 

The major conslusion reached was that the 
use of the plastic filters did not enhance range 
firing accuracy. 


Received August 9, 1949 





Evaluation of Aircraft Instrument Displays for Use with the 
Omni-Directional Radio Range (VOR) * 


A. C. Williams, Jr., and S. N. Roscoe 
University of Illinois 


The omni-directional radio range (VOR) 
provides the pilot with continuous visual infor- 
mation concerning his bearing to or from an 
omni-range station. This makes it possible 
to fly to a station along any desired track or 
from a station in any desired direction, repre- 
senting a considerable advantage over the con- 
ventional four-leg auditory radio range. In 
addition, VOR employs a very high frequency 
signal which provides static-free reception. It 
is, however, limited to line of sight distance. 

The purpose of the present study was to 
evaluate selected aircraft instrument displays 
which could be used with VOR. The evalua- 
tion was not based on engineering criteria, 
since the human engineer is not primarily con- 
cerned with the operation of the machine as 
such, but rather with the performance of the 
man operating the machine. Thus the de- 
pendent variables which were measured and 
compared were the speed and accuracy with 
which representative pilots could use the dis- 
plays in question. An inspection of existing 
VOR instrument displays was sufficient to 
suggest that pilots would not be able to use 
these instruments either quickly or without 
frequent errors because of the inherent ambi- 
guity in the immediate information presented.' 

The problem seemed essentially to be one of 
mode of display. Information presented by 
all displays was correct, adéquate and clearly 
legible, but it was often difficult to interpret. 
For this study, three new displays were devised 
which, it was believed, might represent an im- 

* This article is based upon research conducted at the 
University of Illinois, Ur , Illinois, under the aus- 
pices of National Research Council Committee on 
Aviation Psychology, with funds provided by the Civil 
Aeronautics Administration. 

1 Melton, A. W. Psychological problems in cockpit 
instrumentation for the omni-directional range (ODR) and 
distance measuring equipment (DME). Washington, 


D. C.: CAA Division of Research, Report No. 76, 
February, 1948. 


provement in ease of interpretation compared 
with existing displays. For comparison these 
three were evaluated in conjunction with five 
conventional displays of varying design. In 
all, eight displays were tested and compared. 
They are illustrated in Figures 1 to 8, inclu- 
sive? They were: (1) Conventional Symbolic 
Indicator, (2) Air Line Indicator, (3) Air Force 
Indicator, (4) Experimental Symbolic Indica- 
tor, (5) Radio Magnetic Indicator, and (6), (7) 
and (8) Pictorial A, B and C, respectively, 
special displays designed for this study. For 
illustrative purposes Figures 1 and 6 only are 
presented here: 


Fic. 1. The ConventionaL Sympoxic INDICATOR. 
The Course Line Selector reads: your selected course 
is 170 degrees. The Ambiguity Indicator reads: your 
selected course is TO the station. (If you were South 
of the station, it would read “rrom,” indicating that 
your selected course of 170 degrees radiated out from 
the station in that direction.) The Course Line Devia- 
tion Indicator reads: you are to the Lerr of your 
selected course. (Without the Directional Gyro read- 
ing of 168 degrees you would know only your approxi- 
mate position and would have no way of knowing in 
which direction you were flying at the moment.) 

* To reduce printing costs, Figures 1, 2, 3, 4, 5, 6, 7, 
and 8 and Tables 3, 4, 5, 6, 7, and & have been deposited 
with the American Documentation Institute and may 
be ordered as Document 2742 from American Docu- 


. mentation Institute, 1719 N Street, N.W., Washington 


6, D. C., remitting 50¢ for microfilm (images 1 inch 
high on standard 35 mm. motion picture film) or 60¢ 
for photocopies (6 X 8 inches) readable without optical 
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Fic. 6. Pictoriat A. This instrument is a visual 
display representing an area about the VOR station. 
North is always at the top. The station is the small 
circle in the middle of the scope. Your airplane ap- 
pears as an arrow-shaped pip which shows the actual 
spatial relationship of your airplane to the station. 
The arrow head of the pip shows the heading of your 
airplane which can also be checked by referring to the 
Directional Gyro. The Track Selector serves only as a 
reference line indicating your desired track to or from 
the station. 


The information given by the eight displays 
shown is in each case the same. All indicate 
that the airplane’s present position is some- 
where to the North of the VOR station and that 
it is flying on a heading of 168 degrees. On the 


first four displays the airplane’s position is 


inferred from the fact that it is somewhere to 
the left of a hypothetical course of 170 degrees 
to the station. On the others its azimuth 
position is indicated directly. On the pic- 
torial displays the airplane’s distance from 
the station is also shown, although this infor- 
mation is not given by the VOR signal and 
was not necessary for the solution of the prob- 
lems used in the experiment. In all displays 
the airplane’s heading is indicated directly. 

For those unfamiliar with such displays, 
some idea of how they are used and of their 
relative difficulty can be gained by a simple 
exercise. In each case assume that the pilot’s 
task is: To fly to the station along a track of 170 
degrees, and then try to decide how he should 
fly in order to solve the problem. What is 
the first thing he should do? (In each case the 
pilot should first turn right to a heading of ap 
proximately 2/5 degrees. This would cause 
him to intercept his desired track at about a 
45 degree angle, thus putting him in a position 
to turn to the heading of his desired track and 
fly directly to the station.) 

Designs (1) to (5) represent primarily sym- 


bolic modes of display, because they present 
information by means of numerical pointer 
readings, needle deflections, or numbers ap- 
pearing in windows. Designs (6), (7) and (8) 
were intended to be graphic or pictorial dis- 
plays because the actual horizontal spatial 
relations between aircraft, station and heading 
are shown in miniature on the face of the instru- 
ment. In general, symbolic displays provide 
pieces of information appropriate for a cogni- 
tive solution to a navigation problem, while 
pictorial displays provide cues appropriate for 
a perceptual solution to a problem. In this 
case the dichotomy was not absolute, because 
some of the symbolic displays also provided 
cues which could be interpreted perceptually, 
while the pictorial displays provided some 
symbolic quantitative information. 

Considering these facts, an experiment was 
designed to measure and contrast the speed 
and accuracy with which representative pilots 
could use the eight displays in solving repre- 
sentative air navigation problems. 


Description of the Experiment 


The experiment was designed in such a way 
that variability in performance due to indi- 
vidual differences among pilots as well as differ- 
ences among displays could be estimated by 
the analysis of variance technique. The design 
did not provide specifically for an accurate 
estimate of the possible interaction between the 
various displays and the different types of 
navigation problems for which they were used. 

The displays evaluated were picture mockups 
instead of actual flight instruments. The 
mockups were made by drawing schematic 
pictures of the displays, the indications of 
which were made to read as they would on the 
real instruments had the aircraft been in a 
given position with respect to a VOR station. 
A series of ten navigation problems was made 
up based upon ten different positions of the 
aircraft with respect to the VOR station. The 
problems were selected so as to sample a 
variety of flight situations in which VOR might 
be used. Each problem specified a task which 
the pilot was supposed to solve by reading the 
display. The tasks in general were either to 
fly to or away from the VOR station along a 
track through the aircraft’s present position, 
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or to fly to or away from the station on some 
other designated track as might be required by 
traffic control. To indicate his solution the 
pilot was given a double multiple choice of 
answers. On the first part he had to indicate 
whether he would turn right, fly straight, or 
turn left to initiate his solution. On the 
second part he had to choose one of four head- 
ings to which he would turn if in the first part 
he had decided to turn, or he could indicate 
he would maintain the same heading if he chose 
not toturn. For example, if his problem were: 
‘To approach the station along a track through 
my present position,” his double multiple 
choice of answers might be: 
North. 
turn right 90 degrees. 

“T should fly straight to/on a heading of same as present 


turn left 80 degrees. 
270 degrees.” 


All of the answer choices were in this form. 
The subject was required to circle the proper 
words in the first and second parts so as to 
make a complete statement of his solution of 
the problem. 

A set of ten mockups representing the ten 
problems was drawn for each display. The 
problem sets were similar for all displays but 
not identical. The basic spatial relations in a 
given problem remained the same for all dis- 
plays, but bearings, headings, and tracks were 
systematically varied from one display to the 
next so as to prevent pilots from memorizing 
the problems as they went from one problem 
set to the next. The order of problems within 
each set of ten was different for each display 
so as further to discourage learning the 
problems. 

Subjects 


The problem sets were given to 48 pilots 
divided into three groups as follows: Group 
I—16 non-instrument pilots with approxi- 
mately 100 hours of flight experience; Group II 
—16 commercial pilots with CAA instrument 
ratings, most of whom were also flight in- 
structors; Group III—16 scheduled airline 
pilots.’ 

* The authors wish to express their appreciation to 
the 16 United Airline pilots who served as subjects 
and to Col. A. D. Tuttle and Dr. G. J. Kidera who 


recruited their services and arranged for the testing 
sessions 


Procedure 


In order to introduce the pilot-subjects to 
their task, it was found necessary to prepare 
elaborate written instructions supplemented by 
verbal briefing concerning the displays and the 
types of problems to be solved. A set of 
general instructions covering all displays and 
problem types was issued. These were fol- 
lowed by a specific set of instructions for each 
display. These written instructions were too 
extensive to be included here. However, they 
described as clearly and completely as possible 
the interpretation and use of each part of each 
display and the specific flight procedures which 
would be followed in solving each type of 
problem.‘ 

The problems and procedure were pretested 
using a separate group of eight subjects. Asa 
result of the pretest it was found necessary to 
change some of the mockups because of pre- 
viously undetected errors in their readings. 
Likewise it was found necessary to rewrite 
many parts of the instructions because they 
were not understood by the pre-test subjects. 
In general the instructions had to be amplified. 


Experimental Design 


Since all displays could not be used simul- 
taneously by all pilots, it was necessary to 
arrange the sequences in which they were used 
by different pilots so as to balance the effects of 
practice, transfer and fatigue. Since there 
were eight displays, it was necessary for them 
to be used in eight different serial sequences. 
By virtue of such an arrangement, each display 
could be presented once and only once in each 
serial position while also appearing only once 
in each of the eight serial sequences. Two 
subjects from each group, or six subjects in 
all, were assigned to use the displays in an 
order according to each sequence. 

How well displays could be used by the pilot- 
subjects was measured by the time required by 
a subject to complete each set of ten problems 
and also by the number of incorrect answers 
in each set. A problem was considered in- 

‘Complete reproduction of these instructions may 
be found in Williams, A. C., Jr., and Roscoe, S. N. 
Evaluation of aircraft instrument displays for use with 
the omni-directional radio range. Washington, D. C.: 


CAA Division of Research, Report No. 84, March, 
1949. 
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correctly solved if either part of the double 
answer were wrong. 


Results 

1. Comparison of the Groups. Before any 
data pertaining to differences among displays 
were analyzed, both the time and error scores 
for the three groups were tested for homo- 
genity by use of the /-test. The three groups 
did not differ significantly in their time scores. 
With respect to error scores the groups were 
different, both of the more experienced groups 
making significantly fewer errors than the non- 
instrument group. Because of this difference 
the three groups will be treated separately in 
the evaluation of the displays. 

The speed and accuracy with which the 
various pilot groups used each display are 
shown in Table 1. The time scores represent 
the average time required to complete each set 
of ten problems. The error scores represent 
the average number of problems incorrectiy 
solved per set of ten. 

There was a strong relationship between the 
speed and accuracy with which the three groups 
used the various displays. This is shown by 
the rank order correlation coefficients listed in 
matrix form in Table 2. These correlations 
indicate that displays which were used more 
rapidly also tended to be used more accurately. 
This relationship obtained for all groups of 
subjects, even when the time scores for one 
group were correlated against the error scores 
for a different group. 

2. Comparison of the Displays. Table 1 
shows considerable variability among the time 
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and error scores for the different displays. For 
example, using Pictorial A, the airline pilots 
required on the average six minutes to solve 
ten problems, of which only 1.5 were wrong; 
whereas they required 21.1 minutes and made 
4.8 errors per ten problems using the Air Force 
Indicator. Similar ranges were found for all 
three groups. 

As a preliminary test to determine whether 
these apparent differences between displays 
were significant enough to warrant more de- 
tailed treatment, analyses of variance were 
made for the raw speed and error scores of the 
16 subjects in each of the three groups. The 
design for these analyses was categorical by 
displays and subjects. Thus, variance be- 
tween displays and variance between subjects 
were compared with the residual variance. 
Since error scores on individual problems were 
dichotomous, i.e., each problem was scored 
either right or wrong, and since time measures 
for individual problems were not obtained, it 
was not feasible to categorize the data in terms 
of problems. Summaries of the six analyses 
are given in Tables 3 to 8, inclusive, which are 
deposited in ADI (see footnote 2). 

These six analyses show the results to be 
consistent for all three groups. There were 
in all cases significant amounts of variance (at 
the 1% level of confidence) attributable to 
differences among the displays. Practically 
all of the variance was accounted for in terms of 
two sources: (1) differences between displays 
and (2) differences between subjects. 

On the basis of the indicated differences 
among the displays, individual comparisons 


Table 1 


Average Time and Error Scores per Set of Ten Problems for Each of the Eight Displays Tested 
(Time scores are in minutes) 


Group I 


Time Errors 


2.8 
4.4 
48 
5.8 
6.3 
6.6 
6.0 
69 


Display 
Pictorial A 
Pictorial B 
Pictorial C 
Radio Magnetic Indicator 
Air Line Indicator 
Air Force Indicator 
Experimental Symbolic 
Conventional Symbolic 


7.9 

9.5 
10.0 
14.2 
17.2 
20.3 
14.2 


Group IIT 


Errors 


Group II 


Time 
5.6 
64 
&.5 

15.4 
15.4 
19.9 
13.4 
194 


Errors 
0.9 
2.8 
28 
4.0 
4.1 
3.8 
43 
43 
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Table’ 2 


Matrix of Rank Order Coefficients Between Time and 
Error Scores for Different Displays by 
the Three Subject Groups 


Variables: T; = time scores for Group I, E; = error 
scores for Group I, Ty = time scores for Group IT, etc. 


Ey Tm Eun 


94 
83 
95 


were made by the /-test for correlated means. 
The error scores for each group on each display 
were pitted against the corresponding scores for 
each of the other displays. A similar treat- 
ment of the time scores was not made. How- 
ever, similar differences would be expected to 
obtain in both cases, since the time and error 
scores were known to be highly correlated (as 
shown in Table 2). In any event, accuracy 
would seem to be a more critical criterion than 
speed in this particular case. 

The results of these comparisons of the error 
frequencies for the different displays are shown 
in Table 9. The displays are listed at the left 
according to their rank order based on error 
scores. Across the top they are listed in the 
reverse order. Any given percentage appear- 
ing in the matrix refers to the level of signifi- 
cance of the superiority of the display indicated 
to the left over the display indicated above. 
The matrix could be constructed in this way 
because there were no reversals in the hierarchy 
of differences. Although a higher ranking dis- 
play was not always significantly better than 
all lower ranking displays (see lower part of 
matrix), a lower ranking display was never 
superior, for any group of subjects, to one 
ranking above it. 

Table 9 reveals the following results. There 
were virtually no significant differences among 
the five displays designated as symbolic. In 
four scattered cases the Radio Magnetic Indi- 
cator was used more accurately than the three 
lowest ranking displays. In three of these four 
cases this advantage obtained only for the air- 
line pilots who may have had some additional 
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experience with some similar type of display, 
for example, the radio compass. 

In general, this matrix emphasizes the di- 
chotomy, discussed earlier, between symbolic 
and pictorial displays. Furthermore, pictorial 
displays should be centered about the station 
(with North at the top) rather than about the 
aircraft, as demonstrated by the superiority of 
Pictorial A over either Pictorial B or C. The 
airplane, rather than the station, should be 
represented by the moving part. 

3. Reliability and Internal Consistency of the 
Test. Inasmuch as the experimental technique 
employed consisted of a paper and pencil test, 
such a test should meet certain standards of 
reliability and internal consistency. 

No direct reliability data were available 
since a single form of the test was given only 
once to any one subject, and since the indi- 
vidual problem sets were hardly long enough 
to employ the split half method. However, 
the reliability of the test may be inferred to 
be high on the basis of the consistency of the 
results for the three independent groups of 
subjects (see especially Tables 1, 2, and 9). 
Furthermore, an item analysis showed the 
test to be internally consistent, since each of 
the ten test items or problems correlated 
acceptably with the total test error scores for 
each of the three groups. On the average the 
individual items correlated about .60 with the 
total test. (Time scores were not obtained 
for the individual items. ) 

4. Dependence between Problems and Dis- 
plays. It would be of interest to determine 
whether some instruments are relatively more 
effective than others in working certain types 
of problems while being less effective for others, 
i.e., whether there was any “interaction” be- 
tween instruments and problems. Unfortu- 
nately, however, the data are not such as to 
yield a direct answer to this question. Such 
interaction could not readily be evaluated by 
analysis of variance procedures for reasons 
stated previously, namely, error scores for in- 
dividual problems were dichotomous and time 
scores for individual problems were not ob- 
tained. Nor were the assumptions required 
for the use of certain other statistics, such as 
chi-square, completely justified. Inspection of 
the raw error data suggested, however, that 
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Table 9 


Matrix of Significant Differences in Error Scores for Groups I, I] and III on the Different Displays 
Percentages indicate the level of significance of the superiority of the display indicated at the left over the 
display indicated at the tp, as determined by the ¢-test for correlated means. The single division line emphasizes 
the dichotomy between the “pictorial” and the “symbolic” displays. The double division line emphasizes the 


superiority of the station centered pictorial display over all others, including the airplane centered pictorials. 


Composite 
Error : ; 
Rank Display Group CSL. E.S.I. AFL. ALI RMI. PicC Pic B Pic A 
I A% N% AQ% AW% A% 2% 2% 
1 Pic A ul W% A% N% 1% 1% 1% 1% 
iil A% A% 1% A% A% 


A% 5% A% 2% 2% 
2% 1% 5% 1% 
A% A% A% A% 2% 
1% 
S% % 
1% 

/© 


there were no marked trends indicating de- experimental task and the flight task. In the 
pendence between problems and displays. use of any such instrument display, the pilet’s 
Since a wide variety of possible flight situations task involves two aspects of performance, one 
was sampled by the ten problems, this apparent of discrimination in which the pilot must 
lack of interaction is encouraging in that it decide what to do, and one of manipulation in 
further justifies making generalizations con- which he must execute his decision. 
cerning the relative overall eflectivences of the In this experiment, it was possible to get at 
various displays for air navigation. : Pent OF ty Fo 
only the first aspect of the pilot’s task, namely: 
his decision concerning which way to fly in 
order to solve the navigation problem at hand. 
For this purpose it is believed that the results 
displays can be used to solve navigation prob- ate, pecrmens Se the flight situation. ; On the 
lems with greater speed and accuracy than can other hand, by one. See kups some informa- 
symbolic type VOR displays. Whether these tion was lost which could have been obtained 
results are applicable to flight depends upon the in flight by watching the trend of the instru- 
significance of the discrepancies between the ment readings. But this handicap would 





Discussion of Results 


Within the scope of this experiment the 
results have shown that pictorial type VOR 
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apply equally to all displays and would prob- 
ably not affect their relative standing. 

A second disadvantage of mockups is that 
the subject could not manipulate the instru- 
ment so as, for example, to change a ‘‘to” 
course to a “from” course or vice versa, as 
could be done with some of the actual instru- 
ments. At the same time the opportunity to 
manipulate an instrument is also an oppor- 
tunity to make an error in manipulation as 
has been found to be the case in a Link trainer 
equipped with VOR. Even though the in- 
ability to manipulate the mockup might 
handicap some of the symbolic displays, the 
same handicap existed for the pictorial dis- 
plays in the case of their track selector which 
could not be manipulated on the mockup but 
could on the rea! instrument. For this reason 
it is doubtful that lack of manipulation dis- 
criminated unfairly between the various types 
of displays. On the whole it is the opinion of 
the investigators that so far as the pilot’s 
initial task of orientation and discrimination 
is concerned the results of this experiment 
would recur in the actual flight situation. 

After deciding what to do as a result of 
reading and interpreting a display, the pilot 
must then execute his decision by flying the air- 
plane accordingly. To do this the pilot uses 
additional instruments such as the artificial 
horizon, altimeter, etc. But he also uses the 
VOR display as a source of direction informa- 
tion along with the compass and directional 
gyro. When used in this way the display can 
be considered to be a flight instrument and, 
as is true in the case of other flight instruments, 
the pilot’s task then becomes chiefly one of 
alignment. Using the aircraft controls the 
pilot must manipulate the airplane so as to 
align a moving indicator of some sort with a 
previously determined fixed reference on the 
display. 

This sort of performance cannot be evaluated 
by use of static mockups because there are no 
controls and no moving parts involved in the 
mockups. It would be unfair and unwise to 
suppose that the results of this experiment 
could be used to infer the adequacy of these 
displays in their capacity as flight instruments. 
To the contrary there is reason to believe on 
theoretical grounds that a more accurate flight 
path can be achieved using some of the sym- 
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bolic displays than by using the pictorials. 
Symbolic displays using the course line devia- 
tion indicator are more sensitive to displace- 
ment from the selected track than are the 
pictorial displays. Both the Air Force Indi- 
cator and the Experimental Symbolic Indicator 
were designed so that an asymptotic approach 
to a selected track could be achieved by main- 
taining a simple alignment on the face of the 
display. Presumably the ease of accomplish- 
ing this performance using these displays 
would not be surpassed using pictorial displays. 
Certainly an evaluation based on mockups is 
not relevant in this respect. 

On the whole it is accurate to say that this 
experiment has thrown no light on the ade- 
quacy of the displays when they are used as 
flight instruments. There is reason to suspect 
that those displays which were found in the 
experiment to be superior would not necessarily 
be superior when used in this manner. 

It must be remembered, however, that VOR 
is primarily a navigation device. Its chief 
purpose is to supply the pilot with information 
from which he can orient himself and from 
which he can decide on a proper direction in 
which to fly. If the VOR display fails in this 
respect so that the pilot makes wrong decisions, 
then whatever excellence it might have as a 
flight instrument is wasted. If it were a case 
of the pilot doing the wrong thing very well as 
opposed to, perhaps, doing the right thing 
passably well, the latter alternative is to be 
preferred. This experiment has contributed 
evidence bearing on the inclination of pilots 
to make correct or incorrect decisions as a result 
of using different kinds of VOR displays. For 
this reason it is felt that the experiment is 
pertinent to the problem as it exists in flight, 
and that the experimental! results should be 
considered when making a choice of displays 
for actual use. 


Summary 


The speed and accuracy with which 48 pilots 
could use mockups of eight different VOR air- 
craft instrument displays to solve typical navi- 
gation problems were measured. The pilot 
group was composed of 16 non-instrument 
pilots, 16 commercial pilots with instrument 


rating, and 16 scheduled airline pilots. The 
instrument and airline pilots made fewer errors 
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than the non-instrument pilots but there was 
no significant difference in time scores. The 
rank order of displays based on error scores 
was highly correlated with their rank order 
based on time scores both within each pilot 
group and between groups. 

With respect to both time and error scores 
there were significant differences between dis- 
plays, similar differences being found for all 
groups. So-called pictorial displays which pre- 
sented information in terms of a graphic 
representation of the actual spatial relations 
involved were significantly superior to the so- 
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called symbolic displays which presented infor- 
mation in terms of dial readings, needle detlec- 
tions and numbers. One pictorial display was 
superior to all other displays. 

The reliability of the techniques used was 
inferred to be adequate from measures of in- 
ternal consistency and from the similarity of 
results obtained from independent groups. 
Inspection of the data suggested that no 
marked trends indicating dependence between 
problems and displays were evident. 


Received July 8, 1949. 
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The Prediction of Persistency in Premium Payment 


S. Rains Wallace, Jr., and Alfred G. Whitney 


Life Insurance Agency Management Association, Hartford, Connecticut 


This is a report of an attempt to develop 
methods for predicting life insurance premium 
paying behavior. While the research was ini- 
tiated for a purely practical purpose, it offers 
findings of interest to the academic psychol- 
ogist, since it demonstrates that better than 
chance prediction of a response which has a 
quite real social and economic meaning to the 
individual can be made over a period of five 
years on the basis of some simple and frag- 
mentary background data. 


The Problem 


Most life insurance companies (the com- 
monly held belief to the contrary) lose money 
if a policyholder fails to continue his premium 
payments for at least two or three years. 
Even subsequent terminations of the policy 
are costly to the individual agents and agency 
managers, and it can be demonstrated that a 
high policy termination rate in a company, 
other factors being equal results in an un- 
healthy financial condition. Furthermore, the 
companies are not unaware that the problem 
has deeper sociological implications. The 
termination of a policy almost always means 
loss to the policyholder, both financial and 
psychological. It represents a failure in a 
plan for security and, to some degree, an 
implication that the company has been inade- 
quate in meeting the problems and needs of 
its client. For these reasons, the companies 
are eager to find factors which are associated 
with persistency in premium payment and 
which could be employed in training their 
agents how to identify potentially persistent 
clients. Of course, efforts are also made by 
the companies to conserve policies already 
sold, but a previous study (3) has indicated 
that the persistency of a policy is much more 
susceptible to the factors which occur at the 
time of sale than it is to efforts made subse- 
quently to keep it in force. 

Since considerations of selling technique 
make the close catechization of a potential 


buyer something less than desirable, it was 
necessary to limit the study to the information 
which is routinely obtained in the standard 
policy application form. Furthermore, some 
of this information is almost certainly un- 
reliable (e.g., income, occupation, etc.) since 
it is based upon estimates or classifications 
made by the agents rather than actual records 
or the client’s report. The problem, then, con- 
sists in determining methods for weighting 
these items of information in order to obtain 
the maximum degree of accuracy in predicting 
the eventual fate, i.c., the future persistency, 
of a policy at the time it is applied for. 


Methods and Procedures 


The Sample. The basic data for the study 
were first gathered in 1942 (2). Fifty-two 
life insurance companies supplied information 
on 12,499 Ordinary Life policies issued in May, 
1942. Each company was assigned a quota 
of policies on the basis of the amount of new 
life insurance sold in 1940, selected in ac- 
cordance with random sampling procedures. 
Various checks on certain aspects of the sample 
have supported the hypothesis that it may be 
regarded as typical of all Ordinary life in- 
surance sold during May, 1942. 

Five years later, the companies provided in- 
formation on the subsequent history of almost 
all of these policies. One hundred twenty- 
seven of the cases were lost (every company 
contributed to the follow-up and the lost cases 
were distributed throughout the entire sample) 
so that the sample was reduced to 12,372. Of 
these, 1861 policies sold on the lives of persons 
under the age of 15 (so called “Juvenile” 
insurance) were removed. Among the adult 
policies, 1243 were refused by the client, thus 
producing a still-born transaction, while 154 
cases were terminated by death, expiry, or 
maturation. These cases were also removed 
from the sample. The remaining 9114 cases 
included 3898 cases sold to men and 1245 cases 
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sold to women by Ordinary agents. There 
were 3971 cases sold by Combination agents. 

For the purposes of this paper, only the study 
of the 5143 policies sold by Ordinary agents 
will be reported. However, it may be stated 
that similar methods were applied to the other 
group and similar results were obtained. 

The Data. For the original sample, each 
company provided the following information 
for each case: (1) the insured’s annual income 
(usually as estimated by the agent); (2) the 
amount of the policy; (3) whether the insured 
underwent a medical examination ; (4) the type 
of policy (whole life, endowment, term, etc.); 
(5) the manner of premium payment chosen 
by the client (monthly, quarterly, semiannu- 
ally, or annually); (6) the beneficiary (wife, 
children, parent, business partner, etc.); (7) 
the sex of the insured ; (8) the age of the insured; 
(9) the marital status of the insured; (10) the 
occupation of the insured; (11) the amount of 
life insurance owned by the insured; and (12) 
the company or companies in which life in- 
surance was owned. 

For the follow-up sample, each company 
provided the following information for each 
case: (1) whether cash was received with the 
application; (2) whether the policy was issued 
as applied for or at some other rate, type, or 
amount; (3) the number of fractional years’ 
premiums paid as of July, 1947; and (4) the 
status of the policy as of July, 1947 (in force, 
lapsed, surrendered, death, expiry, matura- 
tion, etc.). 

The Analysis. Preliminary inspection of the 
data: for males indicated that students should 
be treated as a special group. The elimination 
of them, in addition to those not gainfully 
employed and those who made single premium 
payments, reduced the sample to 3448. 

For this remainder of 3448, a list of variables 
was studied and persistency rates 


(Persistency Rate 


number of policies in force 
des als seoaral EES 
no. of pol. in force + lapses + vol. term. 


were calculated for various subgroups. A 
summary of the chief relationships which were 
discovered is as follows. There is a tendency 
toward a higher degree of persistency among 
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those: (1) who have higher incomes; (2) who 
choose to make annual premium payments; 
(3) who are older; (4) whose policies are issued 
on an examined basis; (5) who previously 
owned some life insurance; (6) who buy policies 
other than modified life or term policies; (7) 
who are proprietors, executives, or professional 
workers; (8) who buy policies of larger amounts; 
and (9) who are married. 

Other variables which were studied and 
which exhibited a lesser degree of relationship 
are indicated below. There is a tendency to- 
ward a higher degree of persistency among 
those: (1) whose beneficiary is the wife or 
children; (2) who pay cash with the applica- 
tion; and (3) whose policy is issued as ap- 
plied for. : 

The calculated persistency rates for this 
group are shown in Table 1. With respect to 
some variables the total is slightly less than 
3448 because of missing data. 

Since all of the factors found to be related to 
persistency are interrelated in varying degrees, 
the problem becomes one of ‘‘weighting”—in 
other words, of determining a method for 
combining nine variables in a manner to give 
the highest possible predictive effectiveness. 
If all of the divisions of the nine variables 
could be accurately expressed on numerical 
scales, this problem of weighting could be 
solved by standard statistical techniques. 
Since, however, it is difficult to quantify most 
of the variables, the relationship between each 
pair cannot be expressed simply. 

All of the two-way relationships among the 
nine factors listed in Table 1 were therefore 
studied and tables indicating their nature were 
prepared.' Since these relationships could not 
be readily quantified, it was necessary to resort 
to the procedure of continuous subdivision of 
the groups and finally to study the persistency 
of individuals having certain patterns of the 
nine factors. 

For the purpose of this subdividing, policies 
bought on a salary savings basis were elimi- 
nated, since it was believed that the experience 
from 1942 through 1947 with such policies 
would not be applicable to conditions in 1949. 

The initial sorting of the data was made on 

' The interested reader who wishes to examine these 


relationships in detail is invited to write to the authors 
for copies of these tables. 
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the factors, income and mode of premium 
payment. An analysis made at that point 
showed that within income aad mode of pre- 
mium payment groups, the predictive power 
of amount of policy and marital status dis- 
appeared. Accordingly, these two variables 
were eliminated from further consideration. 
The seven remaining variables were each 
divided into a number of categories as shown 
in Table 2. 

The total number of possible patterns under 
such a system of classification would be the 
product of the numbers of categories,— 
4X4K 2K 3K 2K 4K 6= 4008. 


Table 1 


Relation Between Certain Factors and Persistency: 
Sales to Adult Males (excluding students) 
by Ordinary Agents 


Number 
of 


Variable Cases 


Income 
$5000 and over 
$3000-$4999 590 
$1500-$2999 1739 
Under $1500 704 
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3448 


Mode of Premium Payment 
Annually 
Semiannually 594 
Quarterly 
Monthly 
Salary savings 


Medical Basis 
Examined 
Nonmedical 


Previous Ownership of Insur. 
In same company 
In some other company only 
None 


Type of Policy 
Whole life 
Endowment and retirement income 
Family income 
Limited payment life 
Modified life and term 
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___Table1—Continued 


Persis- 
Number tency 
of Rate 


Variable Cases 


Age 
45 and over 
35 to 4 
25 to 34 
Under 25 


474 
818 
1376 
770 
3438 
Occupation 
Executives, proprietors, 
professional workers 
Semi-executive and semi- 
professional workers 
Agriculture and ranching 
Clerical office workers 
Sales clerks and salesmen 
Factory and mine employees, 
skilled and unskilled labor, 
armed forces 


Amount of policy 
$6750 and over 
$3250-$6749 
$2750-$3249 
$2250-$2749 
$1750-$2249 
$1250-$1749 
Under $1250 


Marital Status 
Married 
Single 
Widowed or divorced 


It was decided to make the final sort on 
occupation because it was believed that the 
data on this variable were probably the least 
reliable. Accordingly, by the time this sort 
was made, it was frequently found that much 
of the predictiveness of occupation had disap- 
peared so that, at most, a classification into 
only two or three categories was required. 
For other factors, too, preliminary analysis 
often disclosed lessening of their influence at 
various points. For example, the differences 
in persistency rates among semiannual, quar- 
terly, and monthly business disappeared in 
the highest income group and combination of 
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Table 2 
Initial Variables and Categories 


Number of 


Variable 
Income 
Mode of premium payment 
Medical basis 
Previous ownership of insurance 
Type of policy 
Age 
Occupation 


the data for these three modes of payment was 
possible. By such combinations, the total 
number of patterns was reduced from 4608 to a 
workable number with sufficient cases for 
study in each pattern. 

In each of the separate patterns, the per 
cent of policies in force was calculated and the 
per cents were then converted to a single digit 
numerical rating using the equivalents shown 
in Table 3. 

The data were now examined to determine 
fundamental trends and where inconsistencies 
in these trends could be ascribed to chance, the 
data were smoothed in accordance with the 
trend. A description of the method of per- 
iorming this smoothing can be obtained by 
request to the authors. 


Results 


This analysis, then, resulted in a Persistency 
Rater with which a score could be assigned to 
each case.*. The cases in the study were then 
scored and the actual persistency determined 
for each rating group with the results shown in 
Table 4. The biserial coefficient of correlation 
between the Persistency Rating and the actual 
five year persistency is shown for each group. 

It must be remembered, of course, that such 
“validation” of the Persistency Rater is fal- 
lacious since it is based upon the same cases 
with which the weights were determined. 
Some shrinkage should be expected when the 
Rater is applied to another sample. However, 

* A correction was made in the Rater for the changes 
in income which have occurred since 1942. This cor- 
rection was based upon comparative frequency distri- 
butions of incomes among consumer units in 1942 and 
1948 and upon the comparative values of the dollar in 
1942 and 1949. The Rater, which is now being used 


by a number of companies, may be procured upon 
request to the authors. 


the writers expect such shrinkage to be slight 
in light of the objective nature of the factors 
used. This expectancy has been considerably 
reinforced by the results of a recent study made 
by one life insurance company. 

This company selected, at random, 489 cases 
sold in the first six months of 1946 and studied 
their persistency for a two-year period. The 
sample is not strictly comparable to the larger 
one here reported since it comes from a 
different year and since the over-all persistency 
of the group is considerably higher (82% in- 
stead of 66%). However, when these cases 
were scored on the Persistency Rater, the results, 
as shown in Table 5, indicated a very satis- 
factory level of validity. 

Discussion 

It is not believed necessary to labor the point 
that the discontinuance of life insurance pay- 
ments is a response which has many implica- 
tions for the individual in question. Equally 
apparent would seem to be the possibility that 
such a response might have considerable sig- 
nificance to the psychologist, particularly in the 
social field. While the motivation behind the 
buying and subsequent payment of an insur- 
ance policy is certainly something less than clear 
or simple, it does appear to involve some as- 
pects of “altruism,” of “social responsibility” 
and of “economic stability.”” This study dem- 
onstrates that the response itself must have 
considerable reliability even despite the fact 
that we may identify numerous factors which 
might affect it and which are largely inde- 


Table 3 


Conversion Table for Persistency Rates and 
Numerical Ratings 


Numerical 
Rating 


Per Cent 
in Force 


100 9 
90-99 

80-89 

70-79 

58-69 

48-57 

38-47 

28-37 

18-27 

Under 18 





Prediction of Persistency in Premium Payment 


Table 4 


pean 4 of Placed Policies Which Remained in Force for Five Years and nee Rawes 








All Men Except 
Students 


Per Cent 
in Force 


Persistency 
Rating — No. 


7 and over 808 85% 
6 7 74 
5 736 62 
4 2: $2 
3 and under 461 35 
3209 
Tus = 46 


Total 


pendent of the individual (economic readjust- 
ments, changes in marital status, job changes, 
etc.). The response would, therefore, appear 
to offer considerable promise of providing a 
criterion for investigations in the fields of 
personality or social psychology. While many 
of the factors used in predicting the response 
may be regarded as reflecting the transaction 
rather than the buyer’s characteristics, they 
need not be so interpreted. For example, the 
fact that policies paid for on an annual basis 
are more persistent may indicate the superi- 
ority of this method of payment, but they may 
indicate, instead, that persons who are capable 
of or who choose such a method are more 
likely to make the desired response. The 
latter hypothesis gains some measure of support 
from the fact that persons who change from 
an annual premium payment to a more fre- 
quent one have almost as high a persistency as 


Table 5 


Proportion of Placed Policies Which Remained in Force 
for awe Years and Persistency Rating 








Sales by One Company to All Men Except Students 
Persistency Per Cent 
Rating Number in Force 


. 68 100% 
7 142 91 
6 103 81 
5 
4 





78 75 
71 


3 and under 50 


Total 82% 


Male Students _W omen 


~ Per Cc ent 
in Force 
86% 
75 
62 
52 
38 


“Per Cent 
in Force 


1115 717% 
Tous * 36 


76% 
Tos = 30 


those who continue the annual mode of pay- 
ment (1). Here, apparently, the interest of 
the individual and willingness to go through 
the process of changing his mode of payment 
outweighs the fact that he has more frequent 
opportunities to make the undesired response. 

A factor which could not be included in this 
study but which other studies (3) have shown 
to have considerable influence is the agent who 
makes the sale. It is known that policies sold 
by some agents have a consistently high per- 
sistency, while those sold by others are con- 
sistently low. While it seems probable that 
some of this effect is related to the agent’s 
selection of clients, it appears that some 
characteristics of the relationship established 
between the agent and his client may also be 
important determiners of the client’s future 
responses. It is in this field that the prospects 
for improving accuracy of prediction and for 
enhancing our understanding of the deter- 
mining factors would appear to lie. Studies 
in this direction are now in process. 
Received January 16, 1950. 
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Super, Donald E. Appraising vocational fil- 
ness. New York: Harper and Brothers, 
1949. Pp. xiii+-727. $6.00. 


The primary purpose of the author is to 
provide an objective and detailed evaluation 
of a number of the most important tests em- 
ployed by vocational! psychologists. This has 
been realized in a series of chapters which are 
thorough, straightforward, and with few ex- 
ceptions free from ambiguity. The author has 
summarized for each test the most significant 
research, discussed it in terms of current theory 
and practice in mental testing, and evaluated 
it on the basis of its use and reputation in 
educational guidance, consultation clinics, and 
business and industry. The work is scholarly. 

A second objective of the book is to provide 
the reader with work habits and thought proc- 
cesses which may enable him to evaluate new 
testing instruments and new research. The 
realization of the second, objective does not 
appear to be as complete as the realization of 
the primary objective. Although Dr. Super 
has certainly not limited himself to the cata- 
loguing of tests and their essential statistics in 
the manner which is so prevalent among texts 
in this field, he does not make explicit many 
of the important concepts and psychological 
assumptions which are implicit in the evalua- 
tion of human behavior. He tends to avoid 
the discussion of controversial topics and does 
not strive to illuminate for the student the 
broad psychological basis for and general 
relevance of the samples of behavior elicited by 
tests. His discussion of the basic psychological 
significance of some of the test classifications 
is weak and his appendix on the fundamental 
quantitative principles of mental measure- 
ment is not sufficient to its purpose. Dr. 
Super’s work is critical and intelligent but his 
discussion is no more profound than _ his 
sources. 

The implications of a test score are usually 
discussed in terms of an average relation with 
some isolated criterion and (from the stand- 
point of most clinicians) with insufficient _re- 
gard consistently manifested for such variable 
factors as peculiarities of the individual’s moti- 
vational pattern, his characteristic reaction to 


competition and the probable future enrich- 
ment or impoverishment of his environment. 
Dr. Super’s insightful and stimulating dis- 
cussion of these considerations is familiar to 
those who have read his well-known Dynamics 
of Vocational Adjustment. In his new book, 
however, Dr Super is primarily concerned with 
the portion of the total test variance which is re- 
lated to some known criterion. The possible 
determiners of the residual variance (that 
which makes for errors in predictions) receive 
relatively little attention. The author’s privi- 
lege to make this differential emphasis would 
require no justification and would draw no 
comment had he not in the closing chapters of 
his book discussed the techniques of voca- 
tional guidance and the conversational devices 
whereby tests results are to be presented to 
the individual. 

The book’s greatest value lies in its careful 
delineation of the most important respects in 
which a test may be used. The primary em- 
phasis is on the use of tests. Only the most 
obvious or experimentally illustrated misuses 
of tests are emphasized; the subtle condi- 
tions under which tests may be misused are 
not strongly illuminated. In presenting his 
sources, the writer’s usual critical discernment 
was on a very few occasions in abeyance; and 
although the work is not free from minor 
errors, these are few and do not weaken the 
book. 

In general, Dr. Super’s book comprises .a 
thorough adherence to the literature and he has 
presented its implications for many important 
uses of tests in an expert manner. A pprasing 
Vocational Fitness will be a very desirable 
required text for courses in measurements, 
individual differences, guidance, and vocational 
psychology. 

J. R. Wittenborn 


Yale University 


Bellows, Roger M. Psychology of personnel in 


New York: Prentice- 
Pp. xii+ 499. $4.50. 
This book by an industria! psychologist at- 
tempts to “clarify the boundaries of the 
personnel methods and management field from 


business and industry. 
Hall, Inc., 1949. 
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the socio-psychologica! point of view.” It has 
been written “for those who are interested in 
improving personnel management by use of 
personnel systems and procedures.” 

Part I1—Development of Personnel Tech- 
nology defines the goals of personnel manage- 
ment and describes historical influences leading 
to modern personnel methods. Part II—Tools 
for Effective Use of Personnel includes separate 
chapters on most of the standard topics found 
in personnel management texts, viz., criteria, 
job analysis, recruitment, interviewing, testing, 
training, job evaluation, incentives, merit 
rating, and turnover. Part I1[—Worker Sat- 
isfaction Through Human Understanding de- 
fines the field of “industrial social psychology”’ 
and discusses the need for and techniques used 
in employee counseling, communication among 
employee levels, attitude surveys, and sugges- 
tion systems. Part [V—Implications of Per- 


sonnel Technology describes the rising pro- 
fessionalization and specialization of personnel 
management and the need for trained leaders 
and summarizes current trends in personnel 
practices as influenced by research in the field. 
Appendix A presents lists of publishers, organ- 
izations, publications, and schools as sources 


of additional information. Appendix B pre- 
sents the Taylor-Russell tables. Each chapter 
ends with a list of selected references. 

Both the commonly-referred-to studies and 
recent literature on the topics discussed are 
fairly well covered. Consideration ef the 
human element is stressed throughout and due 
recognition is made of the importance of the 
social influences impinging upon the indivi- 
dual worker. Topics frequently neglected 
are pointed up, as evidenced by separate 
chapters devoted to recruitment, employee 
suggestion systems, and methods of improving 
communication between management and em- 
ployees. The need for basic research and for 
considering the interaction among the various 
personnel techniques is emphasized through- 
out. That personnel psychology is on its way 
to becoming a profession is evident to the 
reader. 

Unfortunately, the essentially good “‘con- 
tent” of the book is not presented in an equally 
effective manner. The writing is redun- 
dant, tautological, uneven in difficulty, and 
sometimes even ungrammatical. Words and 
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phrases are frequently so loosely used as to 
become almost meaningless. Simple, painfully 
obvious comments are couched in involved 
sentences. Even straightforward statements 
are frequently unnecessarily qualified. The 
following are selected from literally scores of 
possible examples: 


“Since Pile A represented the most favorable 
attitude toward the church, this represents 
a fairly favorable attitude toward the 
church.” (p. 376) 

*. . . the job-analysis approach can be use- 
ful only if job analyses are made and. . . .” 
(p. 426) 

“Managements, in considering whether non- 
directive counseling is profitable, are con- 
cerned with the problem of evaluating it.”’ 
(p. 323) 

“Systematic use of . . 
looked.” (p. 159) 
“Accept his feelings and attitudes and re- 
flect it.” (p. 320) 

“. . . qualifications ...is....”’ (p. 326) 
“The non-directive approach is probably 
best adapted to the severity of the malad- 
justment.””  (p. 323) 

*“ . . success in studying and becoming an 
accountant.” (p. 140) 

“The sample had been picked . . . into one 
of five major criterion groups.”’ (p. 153) 
“It is a project of some size to arrange for 
all employees to meet in a single room, par- 
ticularly in large companies.” (p. 333) 
“Luncheon menus . . . might best be placed 
in the restaurant or in the hallways leading 
to the restaurant. . . . Departmental in- 
formation might be placed in the hallways 
that are used by employees in a_ given de- 
partment.” (pp. 337-338) 

“. . . differences . . . has.” (p. 351) 
“Indirect media for sharing information con- 
sist of results as a by-product of the several 
personnel methods that are discussed in this 
book.” (p. 345) 

“This way of doing does not work so com- 
placently in industry.” (p. 437) 

“(This test terminology ] was found useful 
in designating the different types of predictor 
devices since in this program considerable 
value was found in the use of non-test pre- 
dictors, which were systematically used in 


. are often over- 
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the same batteries as test predictors, when 
found to increase the validities of such 
batteries.” (p. 156) 

“A list of 769 words prepared by a professor 
named Dale... .” (p. 355) 


Possibly the author was trying to ‘‘write 
down” the material to a lower level of under- 
standing than this reviewer assumes necessary. 
Perhaps he himself was trying to apply the 
Flesch techniques which he (rightly) recom- 
mends for use in employee communications. 
If so, this reviewer appreciates the effort but 
still considers the level of presentation as 
vague and wordy rather than clear and precise. 

The usual minor errors and omissions found 
in first editions are present. Figure 1 is in- 
completely labelled. In Figure 25 the term 
“applicants” should be ‘‘employees.”’ On 
page 132, ‘selection ratio” is improperly used, 
although defined accurately in Appendix B. 
Appendix B, incidentally, is referred to in the 
text merely in a footnote. Figure 42 is an 
essentially meaningless figure purporting to 
show “The Three Degrees’’ of social distance 
between worker and boss. Table 29 is pre- 
sented with no reference to it in the text. 
Table 40 should be numbered Table 39. The 
format of Appendix A is such that it is quite 
difficult to separate one item from another in 
the long lists presented. Many of the “Se- 
lected References” at the end of the chapters 
do not indicate the relevant section of the 
reference. The addresses of several of the 
journals listed in Appendix A are incorrect or 
out-of-date. 

It is easy for reviewers to point out omissions 
of content. As the field of personnel psychol- 
ogy expands, authors are being forced to 
select what materials to include. This re- 
viewer was disappointed, however, in not find- 
ing reference to British and French industrial 
psychologists, to the use of experimental groups 
hired without regard to test scores as an im- 
portant step in test validation research, to the 
work of the Civil Service Commission on job 
Classification, to the effect of the use of em- 
ployment tests on quality of applicant, to the 
types of tests which have been found useful 
in predicting job success, to the T. W. 1. pro- 
gram of training supervisors, to the use of non- 
financial incentives, to data on voluntary re- 
striction of output by workers, and to the 
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Labor-Management Committees of World 
War II. In addition, although scattered ref- 
erences are made, there is no systematic dis- 
cussion of accidents and their prevention, of 
monotony, of music in industry, of fatigue and 
efficiency. 

This reviewer’s over-all judgment is that the 
author has a potentially good book at hand 
but that it needs a careful going-over and 
condensing before it is worth much more than 
a casual reading of Chapter V. Attracting 
Personnel, Chapter XVI. Techniques for Im- 
proving Communications, and Chapter XVIII. 
Employee Suggestion Systems. 


Albert S. Thompson 
Teachers College, 
Columbia U niversity 


Prentice-Hall, Inc. The new cure for whiie 
collar unrest. 1948. Pp. 48. 


This forty-eight page pamphlet is written in 
the breezy style typical of Prentice-Hall reports 
and services for businessmen. It contains an 
introduction and four sections titled: what 
makes white collar workers tick, why white 
collar workers kick, how unions sell themselves 
to white collar workers, and what employers 
can do for white collar workers. Over half of 
the booklet is devoted to the last of these 
sections and covers such diverse matters as 
merit rating, suggestion systems, personal con- 
sideration, company rules, earmarks of a suc- 
cessful supervisor, job evaluation, pension 
plans, getting information to employees, morale 
surveys, etc. Inasmuch as this itemization 
includes only a small number of the many 
items covered, it is obvious that treatment 
must be at a superficial level. This reviewer 
could find nothing to warrant inclusion of “‘new 
cure”’ in the title. Although the booklet is not 
free from factual error and unsupported state- 
ments, it is basically accurate at a simple level. 
It might thus be of value to the “busy execu- 
tive’ who wishes to cover a maximum of 
subject matter in a minimum of time. The 
pamphlet obviously is not intended for person- 
nel psychologists or professionally trained 
personnel and industria! relations directors. 


Clifford E. Jurgensen 


VW innea polis Ges Company 
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Gilbert, Jeanne G., and Weitz, Robert D. 
Psychology for the profession of nursing. 
New York: The Ronald Press Company, 
1949. Pp. x+275. $5.00. 


The authors state that this textbook has 
been prepared specifically for student nurses. 
The purpose is to present principles of psy- 
chology in such a way that the nurse can make 
use of them in understanding the patient and 
in making a successful personal adjustment to 
life and its problems. 

The book is divided into three parts. The 
emphasis in Part I, Fundamental Principles of 
Psychology, is upon individual human behavior 
and the factors which determine jits develop- 
ment. In Part II, Personality, Mental Hy- 
giene and the Normal Patient, the integration 
of all traits in the development of personality 
is discussed; and special problems which have 
to do with the care of children, aged, and 
chronic and convalescent patients are pre- 
sented. Part III, Personality Maladjustments 
and the Abnormal Patient, includes informa- 
tion about psychopaths and other wayward 
types, behavior disorders in feebleminded and 
organically diseased patients, the psychoneu- 
roses and the psychoses. Case studies are 
presented to illustrate various types of ab- 
normalities. The final chapter includes a brief 
presentation of diagnostic procedures and thera- 
peutic techniques. 

In writing this particular text the authors 
have probably attempted to meet the needs of 
those schools of nursing which, up to the 
present time, have tried to cover the entire 
field of psychology in an introductory course 
usually given to students during their first 
year in the school. It is not adapted to the 
needs of schools which are attempting to inte- 
grate psychological principles into every clinical 
subject which is taught, and which, in addition, 
are providing experience in psychiatric nursing 
for all of their students. The material on 
fundamental principles of psychology, integra- 
tion of personality, diagnostic tests and 
therapeutic techniques; which should probably 
receive greatest emphasis in a basic course in 
psychology; is so briefly presented as to be of 
questionable value unless supplemented with 
readings in other texts and periodicals. Yet, 
the references at the end of each chapter in- 
clude only the names of entire books and 
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periodicals, Without more specific references 
to guide her, the student in nursing is unlikely 
to do extensive supplementary reading. 

In the discussion of adjustment the student 
is warned that she will be expected to adjust 
to a regimented, restricted environment where 
good judgment and adult behavior are ex- 
pected. She is told that nursing is hard and 
that she must be equipped physically, mentally, 
and emotionally to meet the hardship. A 
more positive approach, in which the student is 
challenged to think of nursing as an interesting 
and worthwhile profession, would, in the long 
run, probably have a better psychological 
effect than the negative emphasis upon the im- 
portance of adjusting to conditions as they are. 


Helen Nahm 
Duke University 


Hovland, C. 1., Lumsdaine, A. A., and Shef- 
field, F.D. Experiments on mass communi- 
cation. Studies in Social Psychology in 
World War I1, Vol. III. Princeton: Prince- 
ton Univ. Press, 1949. Pp. viii¢ 345. 
$5.00. 


This is the third in a series of four volumes 
reporting the work of the Research Branch 
of the Army’s Information and Education Di- 
vision during the war. Volumes I and II, 
previously reviewed in this journal (33, 609- 
611), discussed the attitude surveys relating 
to the soldier’s adjustment to army life, and to 
combat and its aftermath, carried out by the 
Survey Section of the Research Branch. The 
present volume reports the work of the Experi- 
mental Section which had as its primary re- 
sponsibility experimental studies on mass 
communication. 

An introductory chapter outlines the vari- 
ables investigated and the general nature of the 
film research done by the Experimental Sec- 
tion. A basic distinction is “made between 
studies where the purpose is to evaluate a com- 
pleted product and those where the purpose is 
to investigate variables by controlled varia- 
tion” (p. 4). In terms of this distinction the 
book is organized into two sections, the first 
reporting the evaluative studies and the second 
the studies employing controlled variation. 
“In both kinds of studies the main emphasis 
was on the measurement of changes in knowl- 
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ecge, opinion, or behavior produced by a film 
or other communication device” (p. 5). 

The first three chapters of Part I are devoted 
to the evaluation of the “Why We Fight” 
series of indoctrination films. Chapter 5 
discusses several evaluative studies involv- 
ing an experimental comparison of alternative 
methods of presenting the same material, e. g., 
a sound motion picture and a film strip. 
Chapter 6 analyzes the effects of films on men 
from different intellectual levels. 

In Part II, one chapter is devoted to a study 
of the short-time and long-time effects of an 
orientation film upon factual knowledge and 
opinions, another to a study of the effects on 
opinion change of presenting one side as com- 
pared with both sides of a controversial issue, 
and a third to the effect of audience participa- 
tion during a film strip presentation on learning 
the phonetic alphabet. Chapter 10 is an ex- 
tremely well done bit of writing which neatly 
summarizes and evaluates the work of the 
Experimental Section. Four important ap- 


pendices follow which deal with some of the 
measurement problems encountered by the 
Experimental! Section in their various studies. 

The results reported, with one exception, 
will not be particularly revealing to psycholo- 


gists. We find, for example, that it is possible 
to increase factual knowledge as a result of 
being exposed to films. Some effects are ob- 
served on opinions specifically covered by the 
films, but these effects are not as great as those 
on factual knowledge. Still less is the effect 
of films on opinions of a more general nature— 
opinions which the films were designed to in- 
fluence, but which were not specifically covered 
by the content. We find also that those with 
greater intellectual ability learn more from a 
film than those with less ability; that an- 
nouncing in advance that a quiz will be given 
on the content of a film facilitates learning; 
that audience participation also facilitates 
learning. These results, and most of the 
others not mentioned here, would be predicted 
in terms of current psychological knowledge. 

The exception, mentioned above, concerns 
the prevalent notion among social psycholo- 
gists that the influence of “propaganda” is 
merely the reinforcing of existing beliefs. But 
in a radio presentation study the Experimental 
Section found that “whether a man was 
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initially for or against the stand taken in the 
communication, his opinion tended to be in- 
fluenced in the direction of more acceptance of 
the point of view argued for in the communica- 
tion. In no case was there found a significant 
change opposite to the intended effect of the 
communication among those initially opposed, 
and in all cases where the audience as a whole 
showed a positive change, the change was posi- 
tive among those initially opposed” (p. 269). 

Although the concept of “‘attitude”’ was fre- 
quently used in the first two volumes, the 
authors of the present volume avoid this con- 
cept, preferring to state their discussion in terms 
of “opinions.” | ““This usage reflects in part the 
fact that no satisfactory way was found of dis- 
tinguishing between attitude questions and ques- 
tions of opinion” (p. 265). But statements of 
opinion are distinguished from statements of 
facts in that “opinions are interpretations of 
available facts, which interpretations are 
difficult to verify or disprove directly” (p. 265). 
“Informed” opinions are those based upon 
“valid” interpretations of available facts. 
“Valid” interpretations, in turn, “are those 
which can be shown to be positively correlated 
with an index of ability to make valid inter- 
pretations” (pp. 275-276), for example, educa- 
tional level. “Invalid” interpretations are 
those negatively correlated with this index. 
This distinction leads to the rather interesting 
definition of “propaganda” as “attempts to 
foster ‘invalid’ interpretations” (p. 276). 

That there are difficulties with this anal- 
ysis will be obvious if it is applied to the 
1948 election campaign. Take the statement: 
“Mr. Truman has little or no chance of being 
elected.”” Would agreement or disagreement 
with this statement have represented “‘in- 
formed” opinion? But which would have 
been the “valid” interpretation of the avail- 
able facts? Remember that a “valid’’ inter- 
pretation is one which is correlated with an 
index of ability to make “valid” interpreta- 
tions and that one such index used by the 
Experimental Section was educational level. 
And now ask and answer the question as to 
the relationship between political party affilia- 
tion and educational level, and you should 
also have the answer to the question of whether 
it was the Democrats or the Republicans that 
engaged in “‘propaganda”’during the campaign. 
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Many of the results of the studies are re- 
ported in terms of the average per cent correct 
answers rather than in terms of the simple 
arithmetic average. This can be somewhat 
misleading. For example, Figure 5, page 143, 
reports the average per cent correct answers 
for a control and various experimental! groups. 
Taking only the figures for the control group 
and the movie group, we find that the former 
averaged 39.5% correct answers and the movie 
group 46.6%, giving a difference, apparently 
the result of the movie, of 7.1%. When we 
realize that the test consisted of only 39 items, 
this difference is not as imposing as the per cent 
figures make it appear. The test mean score 
for the control group would be 15.4 and for the 
movie group 18.2, giving a mean difference 
of 2.8. 

It is impossible to do justice to this book in a 
brief review, but let me say that I found it 
highly provocative and stimulating. Few ex- 
perimental studies dealing with such complex 
situations have been as carefully planned, 
executed, and analyzed as those reported in 
this volume. The results, and particularly 
the methodology, should prove of interest to 
psychologists, film producers, and anyone con- 
cerned with audio-visual aids in indoctrination 
or training programs—whether in education 
or industry. 

Nor is the book concerned with just meth- 
odology and the reporting of experimental 
results. These authors have not been afraid 
to speculate, to hypothesize, to indulge in 
theory--this provides stimulating reading in 
an area where theory is largely lacking. Ques- 
tions are not only answered but raised, with 
the result that sufficient research problems are 
suggested to occupy workers in this field for 
some time to come. 

Allen L. Edwards 


The University of Washington 


Mathewson, Robert H. Guidance policy and 


practice. New York: Harper and Brothers, 
1949. Pp. 294. $3.00. 


The growing pains and _ self-examination 
which have characterized the profession of 
psychology during the years since World War 
II have been characteristic, also, of the kindred 
and partly overlapping fields of guidance. 
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In the American Psychological Association, 
various divisions have been struggling with 
definitions and attempting to delimit fields, 
with both clinical and educational psycholo- 
gists feeling that guidance belongs to them 
while the guidance psychologists contend that 
they represent a distinct field. So, also, in 
the broader fields of education and social 
service, the professions of psychology, educa- 
tion, and social work have been staking claims 
to guidance, each claiming it as its own. To 
complicate matters, there has for some years 
been considerable debate among specialists in 
guidance as to just how the term is to be 
defined, just what needs it attempts to meet, 
and just what services it should provide in 
order to meet them. 

Guidance Policy and Practice is an attempt 
to define terms, clarify objectives, examine 
methods in the light of these objectives, and 
so delimit the field. It represents the thinking 
of a guidance specialist who has had long ex- 
perience as a practitioner at both adolescent 
and adult levels (under educational auspices), 
and who has, during the past few years, given 
considerable time to the training of graduate 
students. It is ata strategic time in the devel- 
opment of guidance and of professional psy- 
chology that Mathewson has put his thinking 
on record, 

In Part I Mathewson discusses fundamental 
factors such as psychological and philosophical 
concepts, individual and social needs, institu- 
tional settings, the psychology of the partic- 
ipants, and costs. Part II treats the im- 
plementation of policies in schools and in the 
community. Part III takes up current issues 
such as the scope of guidance, the responsibility 
of education, and the training and role of the 
counselor. Part IV examines the future, 
particularly the trend toward unity or toward 
a broader concept of guidance than has pre- 
vailed in some circles, educational institutions 
as the primary medium of guidance, and the 
development of sound national policy in a 
period of increasing federal activity. 

Although the book is uneven (the chapter 
on The Institutional Setting seems to deal with 
the obvicus, while that on The Role of the 
Counselor is a timely and effective exposition 
of a much more mature approach to counseling 
than, for example, either Williamson’s or 
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Rogers’), it is on the whole thoughtful, penetra- 
ting, and constructive. This reviewer would 
like to focus on two major contributions, and 
to point out some related weaknesses. 
Perhaps the strongest aspect of Mathewson’s 
philosophy of guidance is his reiteration of the 
developmental nature of guidance. He sees it 
as a process of orientation of the individual to 
society and of the development of a mature, 
independent, socialized, self. This approach 
resembies that of many educators, and stands 
in clear opposition to and contrast with that 
of many guidance specialists who came to this 
field from social work, and who view guidance 
as a service for “persons in need . . . experi- 
encing some breakdown in their capacity to 
cope unaided with their own affairs,’ whether 
these be the choice of a college major or re- 
habilitation for employment after injury. 
Mathewson conceives of guidance as primarily 
educational and preventive rather than re- 
medial; he concludes that schools and colleges, 
as institutions with principal responsibility for 
education and development, are the major 
guidance agencies. He recognizes that guid- 
ance also has an adjustive or remedial function, 
but one of the defects of the argument (sur- 
prising in an author who has worked at the 


adult level) is the failure to recognize the fact 
that guidance needs and services do not cease 


at age 22. In the vocational sphere, for ex- 
ample, guidance does not actually cease with 
“early adjustment” (p. 120), but, as geronto- 
logy is increasingly demonstrating, continues 
through life, with new problems arising and 
new services called for at each of the life stages. 
In this reviewer's opinion, then, Mathewson’s 
definition of the scope and agencies of guidance 
is too limited, even though his emphasis is 
sound. 

The second contribution singled out for com- 
ment in this review is the discussion of the 
trend toward unity in the field of guidance. As 
the recommendations of policy committees of 
the National Vocational Guidance Association 
in the ’30s and again last year have shown, and 
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as evidenced further by current committee 
work in the Council of Guidance and Personnel 
Associations, there is an increasing awareness 
of the fact that adjustment involves a total 
personality, that the problem of vocational 
choice, for example, is very much a part of the 
problem of developing a self-concept which is 
in harmony with reality. Mathewson recog- 
nizes the unity of personality, attempts to define 
guidance in broad terms, and forecasts increas- 
ing acceptance of this point of view. But in 
this reviewer’s judgment he makes two mis- 
takes made by many educationists working in 
the field of guidance: he fails to recognize 
sufficiently the complexity of personality, and, 
failing to do this, errs further by (perhaps 
unintentionally) leaving the reader with the 
impression that all the vocational or educa- 
tional counselor needs to do in order to become 
a personal counselor is to recognize the unity of 
personality. To recognize the unity of per- 
sonality and of guidance without recognizing 
the complexity of personality and of society, 
including the world of work, is to encourage 
amateurish work and to lower standards. It 
encourages vocational and educational coun- 
selors to venture into psychotherapy without 
adequate training in that specialty; it en- 
courages clinical psychologists to venture into 
vocational and educational counseling without 
sufficient k.iowledge of opportunities and re- 
quirements in schools, colleges, and the fields 
of employment. 

Despite these defects, Mathewson’s little 
volume is a thoughtful and important contri- 
bution to thinking on the nature, functions, 
and organization of guidance. While it may 
not contain the final solutions to our cur- 
rent problems of definition, concern, and 
jurisdiction, it should be read by all who are 
interested in finding a truly constructive way 
out of our present functional and organizational 
dilemmas. 


Donald E. Super 
Teachers College, 
Columbia University 
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